Heatmap and atlas

ABSTRACT

A dynamic anatomic atlas is disclosed, comprising static atlas data describing atlas segments and dynamic atlas data comprising information on a dynamic property which information is respectively linked to the atlas segments.

The present invention relates to a dynamic anatomic atlas comprisingstatic atlas data and dynamic atlas data. The invention also relates toa computer implemented data processing method comprising the step ofacquiring the dynamic anatomic atlas, a method for generating thedynamic anatomic atlas and a use of the dynamic anatomic atlas. It alsorelates to a computer program, a non-transitory computer-readablestorage medium and a computer.

TECHNICAL BACKGROUND

Physiologic movements like vital movements, conscious and unconsciousmovements and others as well as other time-dependent physical propertieshave not been considered within a known anatomical atlas. The term“vital movement” means that the body parts are moved by vital functionsof the body such as respiration and/or heartbeat. These functions of thebody sustain life. Conscious movements can be consciously controlled,i.e. muscle movements, for example to move a limb. Unconscious movementsare movements which cannot be controlled by will, i.e. the heartbeat.Other physiologic movements for example include movements of body fluidssuch as the blood. Examples of other time-dependent properties includefor example the change of temperature, the change of pressure and thechange of concentration of a given substance.

One idea underlying the current invention is the integration of dynamicinformation described by dynamic atlas data (for example the informationgained using physiologic volume rendering as disclosed inPCT/EP2016/053291 (as described in Annex A) into an anatomic atlas.According to PCT/EP2016/053291 (as described in Annex A), via elasticfusion—of a reference bin of a 4D CT to the remaining bins—for everyvoxel a trajectory is computed which describes a time-dependent changeof position of the individual pixel. Each trajectory can then becorrelated to other trajectories, for example to find out whatstructures move in a similar way as a target structure such as a tumor.The result can subsequently be displayed as a “heatmap” indicating thedegree of similarity of movement of each voxel to a target structure,for example based on the determined correlations.

The dynamic atlas data (e.g. general trajectory, and movementcorrelation) of certain areas or structures obtained in this way shallbe stored in an anatomical atlas, for example as meta data. This enablesthe analysis of newly acquired individual timely resolved patient imagedata (e.g. 4D CT data) using this dynamic information described by thedynamic atlas data.

Aspects of the present invention, examples and exemplary steps and theirembodiments are disclosed in the following. Different exemplary featuresof the invention can be combined in accordance with the inventionwherever technically expedient and feasible.

EXEMPLARY SHORT DESCRIPTION OF THE PRESENT INVENTION

In the following, a short description of the specific features of thepresent invention is given which shall not be understood to limit theinvention only to the features or a combination of the featuresdescribed in this section.

A dynamic anatomic atlas is disclosed. Compared to a static anatomicatlas, this atlas comprises further dynamic atlas data. This dynamicatlas data describes time-dependent physical properties of the atlassegments (for example movement of the atlas segments or a correlation ofmovement of different atlas segments).

The dynamic anatomic atlas can be generated based on 3D image dataenriched with dynamic data, referred to as dynamic DRR, dynamic CT orsimilarity image (see Annex A). The dynamic CT is for example generatedbased on a 4D CT. The dynamic CTs of a plurality of patients are eachmatched with static atlas image data. The dynamic data of a matchedpatient segment (contained in the dynamic CT of the patient) istransferred to the corresponding atlas segment (enrichment of staticatlas with dynamic the data). Since a plurality of patients should beused to generate the atlas, a normalization of the dynamic data shouldbe performed.

The dynamic anatomic atlas can be used to classify a patient into apatient category (age, sex, type of disease etc.) depending on themovement of patient segments. Also, structures in a patient which cannotbe identified as critical in static patient images but which moveabnormally compared to the atlas can be determined, e.g. non-enhancinglung tumors which are attached to a rib. Furthermore, the dynamic datacan be used to divide static atlas segments into subsegments which movedifferently from each other (e.g. subdivision of lung into a pluralityof segments), thereby increasing the accuracy of results obtainableusing the anatomic atlas.

GENERAL DESCRIPTION OF THE PRESENT INVENTION

In this section, a description of the general features of the presentinvention is given for example by referring to possible embodiments ofthe invention.

The method, the program and the system are defined by the appendedindependent claims. Advantages, advantageous features, advantageousembodiments and advantageous aspects of the present invention aredisclosed in the following and contained in the subject-matter of thedependent claims. Different advantageous features can be combined inaccordance with the invention wherever technically expedient andfeasible. Specifically, a feature of one embodiment which has the sameor a similar function to another feature of another embodiment can beexchanged with said other feature, and a feature of one embodiment whichadds an additional function to another embodiment can for example beadded to said other embodiment.

The invention relates to a dynamic anatomic atlas. The dynamicanatomical atlas may be stored as data in a storage device such as anon-transitory storage medium. The dynamic anatomic atlas for example isan atlas which can be used in a way similar to that of an anatomicalatlas. This type of atlas is for example described in the chapter“Definitions”. The anatomical atlas can for example be an anatomicalatlas referred to as “universal anatomical atlas”, generated for exampledescribed in the chapter “Definitions” below and used for example asdescribed in the chapter “Definitions” below. The dynamic anatomicalatlas for example includes information on a dynamic property as will bedescribed below, therefore being referred to as dynamic anatomicalatlas.

The dynamic anatomical atlas for example comprises static atlas data.The static atlas data may be data comprised in an anatomical atlas asdescribed in the chapter “Definitions”. For example, the static atlasdata describes the static position of at least one object, for exampleof at least one atlas segment.

The static atlas data for example describes a plurality of atlassegments. For example, the atlas segments represent anatomical bodyparts. The static atlas data may include information about at least onestatic property of the plurality of atlas segments such as for example aphysical property such as at least one of the geometry (shape and/orsize, volume, length, height, diameter etc.), position, (optical orphysical) density and temperature of each of the atlas segments andpressure, flux (e.g. flow direction, flow rate, flow rate distribution,. . . ), and concentration of a substance (e.g. oxygen) within each ofthe atlas segments.

For the generation of the static atlas data, the geometry of a segmentcan for example be measured in medical images, using hardwaremeasurement tools such as rulers or using common imaging methods such asdigital images of a patient. The position can for example be determinedin the same way. The optical density of an anatomic segment can bemeasured using medical images obtained by transmission imaging methodssuch as X-ray imaging. The physical density of an anatomic segment canalso be measured by such methods but also using commonly availablehardness measurement methods and tools (pressure-measurement,indentation measurement, elasticity measurement etc.). The temperaturecan for example be measured using a thermometer, a thermocamera,thermoresistive surgical tools or else. The pressure can be measuredusing pressure sensors or by determining the deformation of ananatomical region of which the elasticity module is known. Theconcentration of a substance may be detected using a chemicalconcentration sensor such as a ph-meter or using medical imaging methodssuch as functional MRT (fMRT). The flux can also be measured usingmedical imaging methods such as (4D)CT, (4D)fMRT, medical ultrasound(such as Doppler sonography), flow rate meters or else.

The dynamic anatomical atlas for example further comprises dynamic atlasdata. The dynamic atlas data is for example stored as meta data.

The dynamic atlas data for example comprises information on a dynamicproperty. The dynamic property for example describes a (time-dependent)change of a physical property of an atlas segment. The dynamic propertyis for example at least one of the change of geometry (shape and/orsize, volume, length, height, diameter etc.), position, (optical orphysical) density and temperature of each of the atlas segments and thechange of pressure and concentration of a substance within each of theatlas segments. The dynamic atlas data for example comprises informationon at least one dynamic property, for example information on a pluralityof dynamic properties, for example different types of dynamicproperties.

The dynamic properties may be measured in a way similar to themeasurement of the physical property described above for the staticatlas data. Dynamic properties like temperature changes or changes inthe concentration of oxygen in different segments may be determined bymeasuring a time dependent response of a patients' body to a change,like a stimulus, e.g. local and/or external temperature change,application of medication, physical exhaustion etc. The dynamic propertymay be described as a trajectory, for example a trajectory describingthe time-dependent movement of a segment or a trajectory in a planewhich is defined by two different physical properties such astemperature and density.

The information on the dynamic property for example is respectivelylinked to the atlas segments. For example, this information is includedin dynamic atlas data which is stored as meta data associated with theatlas segments. Alternatively or additionally, this information isstored separately from the static atlas data and linked to the staticatlas data using links such as hyperlinks or a table including linkinginformation, i.e. associating each of the information on the dynamicproperty to individual atlas segments. “Respectively” in this contextmeans that the information on the dynamic property is linked toindividual atlas segments, meaning that an individual atlas segment isassociated with individual information on the dynamic property.

For example, for an individual atlas segment, there is a bijectiverelationship between the static data related to this atlas segment andthe dynamic data related to this atlas segment. For example, thisbijectively linked relationship is given for some or for each single oneof the individual atlas segments. For example, the link and the storageof the static and dynamic atlas data is such that, for an individualatlas segment (for example for each one thereof), both the dynamic dataand the static data or only one thereof is extractable and furtherprocessable. This allows for instance the calculation of correlation ofmovements. The extraction is for example directly possible withoutperforming an image analysis or an analysis of a sequence of images dueto the storage of the dynamic data separately from but linked with thestatic data. “Separately” means for example at different storagelocations (e.g. storage sections, for instance a header of a file) andthat both information types (dynamic and static) are separated indifferent files and/or storage locations. For instance, the header ofthe file describes dynamic data and the file describes static data.

For example, this association is performed upon creating or improvingthe dynamic anatomic atlas as will be described below. The meaning that“information on a dynamic property is linked to an atlas segment” or“information on a dynamic property is linked to at least one of aplurality of atlas segments” is meant to cover for example thefollowing: First information on a first dynamic property and optionallysecond information on a second dynamic property is respectively linkedto a respective one of the atlas segments. This is meant to also coverthe case where this link is performed not just for the respective one ofthe atlas segments but also for a plurality or all of the atlassegments. For example, a plurality of information on a plurality of(different) dynamic properties is respectively linked to the respectiveatlas segments. That is, it should be noted that although singular formmay be used in the claims and the description, this formulation alsoincludes the meaning of “at least one”, for example a plural, i.e. incase the claims refer to a singular entity, they may well include aplurality of the entity.

The information on the dynamic property for example describescorrelations between the dynamic properties of different ones of theatlas segments. For example, the information on the first dynamicproperty of a first atlas segment describes a correlation between thefirst dynamic property of the first atlas segment and the first dynamicproperty of a second atlas segment. Each correlation for exampledescribes a correlation between the (same, in the example “first”)dynamic properties of two individual and different atlas segments. Inthis case, for example, the information on the first dynamic property ofa first atlas segment describes a plurality of correlations between thefirst dynamic property of the first atlas segment and the first dynamicproperties of a plurality of other atlas segments, for example all otheratlas segments. Furthermore “plurality of correlations” means that eachone of the plurality of correlations describes a correlation between thefirst dynamic property of the first atlas segment and the first dynamicproperty of one of the plurality of other atlas segments. If there ismore than one dynamic property, the term “first” can be replaced by“second”, “third” and so on depending on the number of dynamicproperties. The correlations may be stored in form of a matrix, an arrayor else and may for example be stored as meta data associated with thestatic atlas data describing the first atlas segment or stored asdescribed above. For instance, each entry in a first one of a pluralityof lines of the 2D matrix describes one correlation to one other atlassegment with respect to a first dynamic property and the entries in asecond line of the matrix do the same for a second dynamic property andso on.

The correlations, for example in the form of a matrix, can for examplebe averaged and/or normalized between different individuals (for examplepatients) and/or between different atlas segments. For example, theinformation on the dynamic property describes correlations between thedynamic properties of a plurality of different ones of the atlassegments, for example all different ones of the atlas segments. Theinformation on the dynamic property described above for a first anatomicatlas segment for example is determined (and for example stored in thedynamic anatomic atlas) for a plurality of atlas segments or for each ofthe atlas segments. The correlation may be determined upon creating orimproving the dynamic anatomical atlas as will be described below.

As a correlation, for example the cross correlation between twotrajectories of different atlas segments P and Q is used. In thisexample, each trajectory may describe the position of an atlas segmentdepending on time. The trajectory may be generated by connecting (andfor example by interpolating) several positions of the atlas segmentwhich differ from one another depending on time. For example, thepositions are connected in a sequence corresponding to the point in timein ascending timely order. Interpolation might be used to smooth theconnecting line of the positions and/or fill in voids (missing datapoints).

Of course, also other time-dependent physical properties are possiblewhich can be described as vectors

(t),

(t) (e.g. deformation or temperature change). A plurality of suchvectors may define the aforementioned trajectory. For example, values oftwo separate parameters x, y at a point in time t can be described by avector

(t) which defines values x(t) and y(t). Several of these vectors

(t) may define a trajectory representing time-dependent changes of thevalues of the parameters x and y—in this case, the trajectory may lie ina plane defined by the first parameter x (e.g. temperature) and thesecond parameter y (e.g. blood flow rate), wherein each point of thetrajectory indicates an amount of the first parameter x and an amount ofthe second parameter y at a given point in time.

The first parameter x may describe the position in a first spatialdirection, whereas the second parameter y may describe the position in asecond spatial direction. A third parameter z may describe the positionin a third spatial direction. In this case, a vector

(t) for example describes time-dependent values x(t) of the first,time-dependent values y(t) of the second and time-dependent values ofthe third parameter. Several of these vectors

(t) for different points in time t₀, t₁, . . . , t_(n) can be combined,forming a trajectory in four dimensions (3 spatial and 1 timedimension). As noted above, the trajectory can be interpolated forsmoothing and for filling in voids (missing data points).

A trajectory can for example be described by global vectors includingall values of the vector

(t) (e.g.

(t) or

(t)) at several points in time (e.g. all point in time for which data isavailable). For example, the global vector,

comprises all vectors

(t) for (n+1) points in time:

(t₀),

(t₁), . . . ,

(t_(n)). For example, all vector values are stored in the global vectoras follows:

={p _(x)(t ₀),p _(y)(t ₀),p _(z)(t ₀),p _(x)(t ₁),p _(y)(t ₁),p _(z)(t₁), . . . ,p _(x)(t _(n)),p _(y)(t _(n)),p _(z)(t _(n))}.

For the generation of the dynamic anatomic atlas (see below), onlyclosed trajectories might be used which alternatively or additionallyexhibit a periodic temporal behavior. For example, the dynamic propertyis dynamic spatial information in the form of at least one trajectory,for example in case the dynamic spatial information comprisesinformation on a change of position of an atlas segment and/orinformation on a change of geometry of an atlas segment.

To determine the correlation between two trajectories, for example aglobal vector

describing the respective trajectory can be used. For example, a globalvector

describing a first trajectory and a global vector

describing a second trajectory can be used to determine the correlationbetween the first and the second trajectory. The correlation between thefirst trajectory described by all values of

(t) and the second trajectory described by all values of

(t) can for example be determined as follows:

${Corr} = {\quad{\frac{\int_{t = t_{0}}^{t = t_{n}}{\left( {{\overset{\rightharpoonup}{p}(t)} - {\overset{\rightharpoonup}{p}\left( t_{0} \right)}} \right)\left( {{\overset{\rightharpoonup}{q}(t)} - {{\overset{\rightharpoonup}{q}\left( t_{0} \right)}{dt}}} \right.}}{\begin{matrix}\sqrt{\int_{t = t_{0}}^{t = t_{n}}{\left( {{\overset{\rightharpoonup}{p}(t)} - {\overset{\rightharpoonup}{p}\left( t_{0} \right)}} \right)\left( {{\overset{\rightharpoonup}{p}(t)} - {{\overset{\rightharpoonup}{p}\left( t_{0} \right)}{dt}}} \right.}} \\\sqrt{\int_{t = t_{0}}^{t = t_{n}}{\left( {{\overset{\rightharpoonup}{q}(t)} - {\overset{\rightharpoonup}{q}\left( t_{0} \right)}} \right)\left( {{\overset{\rightharpoonup}{q}(t)} - {{\overset{\rightharpoonup}{q}\left( t_{0} \right)}{dt}}} \right.}}\end{matrix}}.}}$

Alternatively or additionally, a component-wise or weighted correlationmay be used. For example, all correlations between all trajectories (

(t),

(t),

(t),

(t), . . . ) of different atlas segments (P, Q, R, S, . . . ) arecalculated as a matrix (CM).

The direction Dir of each trajectory, for example of the trajectorydescribed by all values of

(t) of atlas segment P, can also be calculated, for example as follows:

${Dir} = {\frac{{\overset{\rightharpoonup}{p}(t)} - {\overset{\rightharpoonup}{p}\left( t_{0} \right)}}{\sqrt{\int_{t = t_{0}}^{t = t_{n}}{\left( {{\overset{\rightharpoonup}{p}(t)} - {\overset{\rightharpoonup}{p}\left( t_{0} \right)}} \right)\left( {{\overset{\rightharpoonup}{p}(t)} - {{\overset{\rightharpoonup}{p}\left( t_{0} \right)}{dt}}} \right.}}}\mspace{14mu}{or}}$${Dir} = {\frac{\overset{\rightharpoonup}{p}\left( t_{0} \right)}{{\overset{\rightharpoonup}{p}\left( t_{0} \right)}}.}$

The direction Dir can for example be averaged and/or normalized andassociated to and/or stored for the corresponding atlas segment. Thedirection Dir may not be a scalar and may therefore for example beback-transformed into the dynamic anatomical atlas before averagingand/or normalizing.

As noted above, the information on the dynamic property for exampledescribes correlations between the dynamic properties of different onesof the atlas segments. The dynamic properties of different ones of theatlas segments may include a plurality of different types of dynamicproperties. For example, the information on the dynamic property forexample describes correlations between the dynamic properties ofdifferent ones of the atlas segments

The information on the dynamic property for example describes at leastone normalized dynamic property of at least one atlas segment. Forexample, the dynamic property of some or all atlas segments isnormalized between some or all of the atlas segments, for example sothat the dynamic property of some or all of the atlas segments can becompared with the dynamic property of some or all atlas segments. Forexample, the dynamic property of all atlas segments is normalizedbetween all atlas segments. For example, the dynamic property of some ofthe atlas segments is normalized between all atlas segments. Forexample, the dynamic property of some of the atlas segments isnormalized between the some of the atlas segments.

Alternatively or additionally, the dynamic property of some or all atlassegments is normalized between different individuals, whereininformation of the different individuals was used for creating and/orimproving the dynamic anatomic atlas as will be described below. Forexample, the dynamic property of patient segments of a first group(class) of patients is normalized with respect to a common reference. Asthe common reference, a predetermined value and/or trajectory and/orvector and/or else of the dynamic property can be used. Alternatively oradditionally, as the common reference, a certain patient segment of eachof the patients is used (e.g. a rib). In this example, the commonreference is different among patients or patient groups (classes). Forexample, trajectories of a first patient segment of all patients of acertain patient type (class) are normalized with respect to a commonreference trajectory or parts thereof. For example, a maximum value ofthe common reference trajectory and/or a minimum value of the commonreference trajectory are used as normalization values. For example, thetrajectories of the first patient segment of all patients of the certaintype (class) are adjusted so as to have the same minimum and/or maximumvalues as defined by the normalization values. For example, at least oneof the maximum and minimum values of the trajectories in a certainspatial direction may be used as the maximum and minimum values. Ofcourse, other methods are possible for normalization. The normalizationis e.g. performed to make the individual dynamic properties of differentpatients comparable with each other.

Normalization of other dynamic properties can for example be performedusing a common reference as well. The common reference may be apredetermined vector and/or matrix and/or value(s) or may be the changeof the physical property of a certain patient segment or part thereof,i.e. the change of temperature, flux or geometry. For example, thedynamic properties associated with several voxels are averaged to serveas a common reference.

The term normalization in this disclosure does not relate to thereduction of data to a kind of canonical form but relates to thenormalization of values, vectors, matrices and/or trajectories. For thecorrelation and normalization of trajectories describing time-dependentmovement, reference is also made to Annex A.

The at least one dynamic property linked to an atlas segment is forexample classified according to patient types. The patient types forexample include one or more of patients with a certain disease, patientswith a certain age, patients with a certain anatomical property (obese,missing organs, deformed organs etc.), patients with a certain gender orelse. For example, the at least one dynamic property linked to an atlassegment is classified according to patient types depending oninformation of individuals from different patient types, for exampleinformation on the at least one dynamic property of an anatomic bodypart of individuals from different patient types, for example theindividuals from different patient types used upon creating and/orimproving the dynamic anatomic atlas.

For example, information on the dynamic property of first anatomicalbody parts of individuals of a certain age are determined upon creatingand/or improving the dynamic anatomic atlas. The certain age is then forexample associated with the information on the dynamic property of thefirst anatomical body parts. In case this information is used uponcreating and/or improving the dynamic anatomic atlas, the at least onedynamic property linked to an atlas segment is classified according tothe patient type of the certain age.

The dynamic anatomic atlas for example comprises information on adistribution of at least one dynamic property. The distribution of theat least one dynamic property may describe a probability of a value orvalues of the at least one dynamic property. For example, thedistribution describes a probability of a value of the at least onedynamic property for a certain patient type. For example, thedistribution describes a value or values of the at least one dynamicproperty for healthy patients, for example a value of probability of acorrelation between two trajectories of two anatomic body parts of ahealthy patient. In this example, if the value of correlation betweentwo trajectories of two anatomic body parts of a patient with a diseasedeviates from the value of correlation between the two trajectories ofthe two corresponding anatomic body parts of the healthy patient, theinformation on a distribution of the dynamic property can be used as anindicator whether the patient with a disease has a disease. In thisexample, it can be determined that the patient with a disease has adisease if the value of correlation between two trajectories of the twoanatomic body parts of the patient with a disease has a probability forhealthy patients below a certain threshold, which probability isdescribed by the distribution of the dynamic property. The distributionmay be determined upon creating and/or improving the dynamic anatomicatlas.

The distribution of at least one dynamic property may describe differentprobabilities of a value of the at least one dynamic property fordifferent patient types (patient classes). The classification accordingto patient types mentioned earlier can be used therefore. For example, afirst dynamical property linked to a first anatomic atlas segment isclassified into a first patient type and a second dynamical propertylinked to the first atlas segment is classified into a second patienttype. This means that a first and a second dynamic property are linkedto the one atlas segment, wherein these dynamic properties areclassified into different patient types. As a consequence, there may beone dynamic property respectively linked to a first anatomic atlassegment for each one of the patient classes. This may also be the casefor a plurality of for all of the atlas segments.

For example, a first patient class representing healthy patients and asecond patient class representing patients with lung cancer can beprovided. As noted earlier, a dynamic property can be assigned to eachof these classes. For example, the movement of lung parts or thediaphragm can be classified according to patient types. The informationon a distribution of the at least one dynamic property, for example themovement of the lung parts or the diaphragm, can be used to furtherincrease an error rate of classifying a patient into one of the patientclasses based on the dynamic anatomic atlas. As noted above, a dynamicproperty of a patient can be compared with the distribution. Dependingon the result of the comparison (i.e. exceed a certain probabilitythreshold), the patient can be assigned to the correct patient classwhich works as an indicator for a disease of the patient. Thedistribution gives more reliable results than fixes values because notall patients in one patient class may exhibit the same amount of dynamicproperty, for example the same amount of movement of the lung parts. Thedistribution can therefore be used as a measure of probability of apatient to fall into one of the patient categories.

The dynamic anatomic atlas for example comprises an atlas segmentsubdivided into atlas subsegments respectively linked with differentdynamic properties while exhibiting the same segment representationinformation. For example, the different dynamic properties are of thesame type of dynamic property but represent different values of the sametype of dynamic property. For example, the atlas segment is subdividedinto two atlas subsegments of the same tissue type (e.g. which have thesame visual appearance in medical image data) or which exhibit the samesegment representation information (which is for example storedassociated with the static atlas data), wherein a first of the two atlassubsegments is respectively linked with a first value and a second ofthe two atlas subsegments is respectively linked with a second value ofthe same dynamic property. Different types of the dynamic property forexample include movement, deformation, temperature change, pressurechange or others as noted above.

The invention also relates to a computer implemented data processingmethod comprising the step of acquiring the dynamic anatomic atlas.

The invention also relates to a computer implemented data processingmethod for generating (e.g. improving or generating from the scratch)the dynamic anatomic atlas. The method for example comprises a step ofacquiring, based on the static atlas data, a static atlas image of theatlas segments. For example, a 3D static atlas image such as a 3D CT or3D MR image is generated based on the static atlas data. For example, a2D static atlas image such as a 2D DRR image is generated based on thestatic atlas data. The generated static atlas image represents one ormore of the atlas segments. Other medical image data may be generated asthe static atlas image. For example, the static atlas image is generatedtaking into account a viewing angle of an imaging device used to obtaina static patient image, for example for the generation of the 2D DRR.

For example, the method further comprises a step of acquiring staticpatient data describing a static patient image of a patient segment. Thestatic patient data may be medical image data, for example 3D CT imagedata, 3D MR image data, 2D X-ray image data or else. The static patientdata may further include information on physical parameters such astemperature, concentration of a substance, pressure or else, for examplein a locally-resolved manner. The static patient image may be an imageof one or more patient segments. The patient segment for example is ananatomical body part of the patient and/or a part of the patient with asimilar tissue structure which is for example represented by the sameintensity and/or gray value in an X-ray image. As noted below, theacquisition may comprise loading of the static patient data, for examplefrom a storage medium. The static patient data may be obtainedbeforehand, i.e. the obtaining of the static patient data, for exampleusing a medical imaging device, is for example not part of the method.

The method for example further comprises a step of acquiring informationon a dynamic property. The dynamic property is for example at least ofthe change of a physical property, like geometry, position, (optical orphysical) density, temperature and volume of the patient segment and thechange of pressure and concentration of a substance within a patientsegment. The information for example is respectively linked to thepatient segment. The patient segment for example is an anatomical bodypart of the patient, for example a segmented part in the static patientimage, for example an anatomical body part having a certain tissuestructure. The term “respectively” means that individual information onthe dynamic property is linked to an individual patient segment, forexample different information on the dynamic property is linked to eachof a plurality or all of the patient segments. For example, theinformation on the dynamic property is obtained from dynamic 3D dataand/or a similarity image and/or a dynamic DRR as described which aredescribed in detail in Annex A and respectively linked to differentpatient segments.

The method for example further includes a step of matching the staticpatient image with the static atlas image. The matching for examplematches one or more of the patient segments included in the staticpatient image with one or more atlas segments included in the staticatlas data. The matching may comprise a step of adjusting the positionof the static patient image with respect to the static atlas image,which step may be performed automatically or manually (e.g. adjustmentperformed by a user). For example, an automatic optimization of theposition of the static patient image with respect to the static atlasimage is performed. The matching may include image fusion, for examplerigid and/or elastic image fusion and may be performed by a matchingalgorithm such as a rigid or an elastic image fusion algorithm. Thematching is for example described by a transformation, for example by atransformation matrix, the transformation for example describing atransformation of a coordinate system of the static patient image to acoordinate system of the static atlas image. The coordinate system ofthe static patient image may be defined by the static patient data andthe coordinate system of the static atlas image may be described by thestatic atlas data, for example as meta information included in the data.

The method may further comprise a step of determining, based on thematching, a corresponding atlas segment corresponding to the patientsegment. For example, the matched static patient image is segmentedusing a segmentation algorithm, for example into different patientsegments which for example have different tissue structures. Thepositions of the different segments of the matched static patient imageare for example compared with positions of atlas segments of the matchedstatic atlas image. Based on the matching, for example using thematching result, for example the transformation, and the determinedposition of each of the segments in the coordinate system of the staticpatient image and the known position of each of the atlas segments inthe coordinate system of the static atlas image, and/or the segmentationresults (for example the degree of similarity between the size and/orshape of the patient segments with atlas segments), a correspondingatlas segment is determined which corresponds to the patient segment.This determination may be performed for a plurality of patient segmentsincluded in the static patient image.

The method may further comprise a step for generating (improving orcreating from the scratch) the dynamic anatomic atlas. This step forexample comprises determining, based on the information on the dynamicproperty linked to the patient segment, the information on the dynamicproperty linked to the corresponding atlas segment. For example, theinformation on the dynamic property is newly linked to the correspondingatlas segment and/or the already linked information on the dynamicproperty is updated (e.g. averaged and/or overwritten). As a consequenceof this step, information on the dynamic property is respectively linkedto the corresponding atlas segment. This step can be performed for atleast one atlas segment, for example a plurality of atlas segments,using a plurality of corresponding patient segments and the informationon the dynamic property respectively linked to the plurality of patientsegments. In other words, dynamic information of a patient is used toenrich the anatomic atlas comprising static atlas data so that itcomprises static and dynamic atlas data as described above.

For example, data of multiple (for example classified, see above)patients may be used in this process to generate the dynamic anatomicatlas. For example, the information on the dynamic property of an atlassegment (which is for example classified according to patient types) maybe determined as an average (value) of information on the dynamicproperty of the corresponding patient segment of a plurality of patients(for example a plurality of patients of a particular patient type).

For this purpose, one or more of an average, a weighted average, amedian, a mean or a distribution of the (value(s) of) information on thedynamic property may be determined based on the information on thedynamic property respectively linked to the corresponding patientsegment of each one of the plurality of patients. Alternatively oradditionally, the information on the dynamic property respectivelylinked to the corresponding patient segment may be normalized beforebeing stored or before being used as information on the dynamic propertyrespectively linked to the atlas segment or before being determined asinformation on the dynamic property respectively linked to the atlassegment. The normalization may be performed as described above, forexample using reference information, for example information on thedynamic property respectively linked to a reference structure of thepatient.

Instead of acquiring the patient images, an anatomic atlas comprisingseveral static atlas images describing physical properties (positions,temperature, pressure, concentration of a substance etc.) of the atlassegments at different points in time may be used. For example, theseveral static atlas images can be used to determine the trajectoriesand thereafter the correlations. In this case, no additional patientimages are necessary. In other words, the patient image data can bereplaced with atlas image data in case the atlas image data describesphysical properties of the atlas segments at different points in time.With respect to such an atlas comprising static atlas images, which canfor example be a universal atlas, it is referred to the chapter“Definitions”.

The method for example comprises a step of calculating correlationsbetween the dynamic properties of different patient segments based onthe information on the dynamic properties linked to different patientsegments for determining the correlations between the dynamic propertiesof different ones of the atlas segments described by the information onthe dynamic property respectively linked to the atlas segments. Thesecorrelations as well as the calculation thereof have been describedabove. For example, correlations between the dynamic property of a firstpatient segment to dynamic properties (of the same type) of differentpatient segments are calculated. These correlations are thereafterdetermined as the correlations between the dynamic properties of thecorresponding atlas segment (corresponding to the first patient segment)and the different corresponding atlas segments (corresponding to thedifferent patient segments). The correlations are for example set and/orstored so as to be described by the information on the dynamic propertyrespectively linked to the corresponding atlas segment. Consequently, aplurality of correlations may be stored for the corresponding atlassegment. This determination may be performed for a plurality or all offthe patient segments to generate the dynamic atlas data comprised in thedynamic anatomic atlas.

Alternatively or additionally, based on at least the information on thedynamic property linked to a patient segment at least one normalizeddynamic property for the patient segment is calculated for determiningthe at least one normalized dynamic property described above. Forexample, the dynamic property of some or all of the patient segments isnormalized between some or all of the patient segments. The some of thepatient segments are for example segments in a predetermined anatomicalregion (e.g. lung, abdomen, region defined by a predetermined distancefrom a reference segment, . . . ) and/or segments which are influencedby the same physical mechanism (e.g. heartbeat, breathing motion,conscious movement, . . . ) which segments can be chosen based on firstpredetermined selection criteria and/or segments which are known to havecomparable dynamic physical properties (e.g. movement in a certaindirection to a certain degree (movement amount below threshold), samecyclic phase of movement (same time constant of cyclic movement), samecyclic phase of change of other physical properties such asconcentration of a substance, flow rate, temperature change and others,. . . ) which segments can be chosen based on second predeterminedcriteria. Alternatively or additionally, the dynamic property of some orall of the patient segments is normalized between different patients.Alternatively or additionally, the dynamic property of some or all ofthe patient segments is normalized with respect to a referencestructure.

For example, a reference structure is identified in each of a pluralityof patients which is used as a normalization reference, wherein staticpatient data and dynamic patient data of each one of the plurality ofpatients is used to generate the dynamic anatomic atlas. As describedabove, the information on dynamic properties of patient segments ofdifferent patients can be averaged, weighted averaged or else todetermine the information on dynamic properties of corresponding atlassegments. The reference structure is for example identified in each ofthe patients which (static and dynamic) information are used to generatethe dynamic anatomic atlas. The reference structure may differ amongstpatient classes (for example amongst patient types).

For example, in case the dynamic property is dynamic spatial informationin the form of at least one trajectory, a reference object with areference trajectory is identified in each of the different individuals,for example a reference anatomical body part, for example a rib. Thetrajectories of other anatomical body parts are for example normalizedfor each of the different individuals using the individual referencetrajectory (which may be different for each of the plurality ofpatients). Several normalization methods may be used for normalization.For example, a maximum and/or a minimum and/or average (e.g. mode, mean,median, weighted average) value of the reference trajectory are used asa reference for normalization, for example a maximum and/or minimumand/or average value of the reference trajectory in a particulardirection (e.g. along the main axis or along a sub axis). For example,only closed loop trajectories, for example only cyclic (timely cyclic)trajectories are normalized. For example, the main axis (the amount ofpositional shift in the main axis) of a reference trajectory may be usedto normalize one or more of the patient trajectories.

The normalization may be performed for patients of different patienttypes (classes) independently and/or differently. For example, thetrajectories of patients of a first type (class) are normalized withrespect to a first reference whilst the trajectories of patients of asecond type (class) are normalized with respect to a second reference.For example, the first patient type defines patients with lung cancerwhereas the second patient type defines healthy patients. For example,the first patient class defines patients who are older than apredetermined age, whereas the second patient class defines patients whoare younger than the predetermined age.

The normalization may be performed for each patient individually. Forexample, the trajectories of each of the patients are normalized withrespect to an individual reference. For example, the individualreference is a trajectory of a certain patient segment of the individualpatient (e.g. the rib). For example, the individual reference is areference which serves as an indicator of a certain movement such as abreathing movement, a heartbeat or else. Since these movements (e.g.breathing movement) may affect other anatomic body parts of a patient(e.g. lung is displaced by breathing movement), the individual referenceserving as an indicator of the movement (e.g. a rib) can be used fornormalization of the trajectories of the other body parts (e.g. thelung) with respect to the movement (e.g. the breathing movement). Thisresults in trajectories which are normalized with respect to a certaintype of movement. That can for example make several trajectories ofdifferent patients which have all been normalized with respect to thesame kind of movement (e.g. breathing movement) comparable, independenton the exact amount of movement which might differ greatly betweenpatients (e.g. older patients breath less air per breathing cycle thanmid-aged patients). With respect to the trajectories and thecalculations based on these trajectories (normalization, correlationetc.), it is also referred to Annex A.

Alternatively or additionally, the information on the dynamic propertyof the plurality of patients may be normalized using a common referencesuch as a predetermined reference which is for example independent ofeach of the patients. For example, a predetermined multiplication(and/or division and/or addition and/or subtraction) factor (and/orvector or matrix) is used to normalize the information on the dynamicproperty of each of the plurality of patients.

Also, at least one threshold value may be used to determine which of theinformation on the dynamic property of the patient segments of theplurality of patients shall be used to generate the dynamic anatomicatlas. For example, information on the dynamic property exceeding acertain threshold may not be used for generating the dynamic anatomicatlas. For example, only closed loop movements (e.g. (timely cyclic)closed loop trajectories) are used for generating the dynamic atlasdata.

The method for example comprises a step of determining atlas subsegmentscorresponding to the dynamic subregions, based on subregions exhibitingdifferent dynamic properties. The different dynamic properties are forexample a dynamic property of the same type, i.e. the different dynamicproperties are different dynamic characteristics, e.g. values,directions or vectors of a certain dynamic property (e.g. differenttemperatures or different direction of the main axis of a cyclictrajectory). For example, certain patient segments may comprise dynamicsubregions exhibiting different characteristics of a dynamic property.In this case, the certain patient segments are for example subsegmentedinto a plurality of patient subsegments, wherein each subsegmentexhibits a different dynamic property. The “different dynamic property”may correspond to a certain range of values of the dynamic property.

Similarly to the method described above with respect to the atlassegments, the information on the dynamic property of each patientsubsegment may be used to determine the information on the dynamicproperty of each corresponding atlas subsegment. For example, the methodcomprises a step for generating (improving or creating from the scratch)the dynamic anatomic atlas by determining, based on the information onthe dynamic property linked to the patient subsegments, the informationon the dynamic property linked to the corresponding atlas subsegments.For example, the same information on the dynamic property is linked toseveral patient subsegments, i.e. depending on the resolution of themeasurement method used to determine the information on the dynamicproperty. For example, several patient subsegments may correspond to thesame atlas subsegment or vice versa. In this case, the information onthe dynamic property of the several patient subsegments may be combined(e.g. averaged (weighted, mode, mean, . . . )) to determine theinformation on the dynamic property of the corresponding atlassubsegment or the information on the dynamic property of the patientsubsegment may be determined as the information on the dynamic propertyof the corresponding several atlas subsegments.

The invention also relates to a computer implemented data processingmethod for enabling an analysis of an anatomic dynamic of a patient. Forexample, the method comprises several steps as described above withreference to the method for generating the dynamic anatomic atlas: astep of acquiring the static atlas data and the dynamic atlas data ofthe dynamic atlas, wherein the static atlas data describes a staticatlas image; a step of acquiring static patient data describing a staticpatient image of a patient segment; a step of acquiring dynamic patientdata comprising information on a dynamic property which information isrespectively linked to the patient segment; a step of matching thestatic patient image with the static atlas image; and a step ofdetermining a corresponding atlas segment corresponding to the patientsegment based on the matching.

For enabling the analysis, the method comprises an additional step. Thisstep for example includes comparing the information on the dynamicproperty linked to the corresponding atlas segment and the informationon the dynamic property linked to the patient segment. For example, thecomparing includes calculating a correlation between the information onthe dynamic property linked to the corresponding atlas segment and theinformation on the dynamic property linked to the patient segment. Forexample, the comparing includes calculating a difference between valuesdescribed by the two information. The comparing may include additionalmathematical steps to compare the two information.

The method may further comprise a step of determining, based on thecomparing (comparison) and based on the information on the distributionof the at least one dynamic property described above, whether thedetermined dynamic property of the corresponding patient segment iswithin a predefined range or not. The predefined range may be a range ofthe distribution of the at least one dynamic property, for exampledetermined by a threshold describing a minimal probability of a certainvalue of the dynamic property, wherein the probability of the certainvalue of the dynamic property is described by the distribution.

For example, the distribution of the at least one dynamic property maydescribe the distribution of values of the at least one dynamic propertyamong the plurality of patients used to generate the dynamic atlas data.In case it is determined that the determined dynamic property of thecorresponding patient segment is within the predefined range it may bedetermined that the corresponding patient segment exhibits a normal(healthy) behavior. In case it is determined that the determined dynamicproperty of the corresponding patient segment is not within thepredefined range it may be determined that the corresponding patientsegment exhibits an abnormal (unhealthy) behavior which can indicate thepresence of a disease.

The determination may be based on the classification of the at least onedynamic property. As described above, a separate distribution may beavailable for each of the patient types. In this example, it may bedetermined that the determined dynamic property of the correspondingpatient segment is not within the predefined range of a first patienttype (which may be healthy) but is within the predefined range of asecond patient type (which may have a particular disease). Thisdetermination may be used to determine the type of the patient and atthe same time whether the corresponding patient segment exhibits anabnormal (unhealthy) behavior or a normal (healthy) behavior.

The method may comprise a step of acquiring the static atlas data andthe dynamic atlas data of the dynamic atlas. As a next step, the methodfor example includes comparing at least one dynamic property associatedwith a certain patient class (e.g. patient type) of the correspondingatlas segment with the dynamic property of the patient segment. Thiscomparison may be performed as described above and for example allows todetermine for which patient class there is the highest similarity forone or more patient segments.

The method may further include a step of determining the type of thepatient. For example, the dynamic property associated with the certainpatient class (e.g. patient type) of the atlas segments is used in thiscontext. For example, a first degree of similarity between the dynamicproperty of the patient segment and the dynamic property of thecorresponding atlas segment which is associated with a first patientclass is determined. For example, a second degree of similarity betweenthe dynamic property of the patient segment and the dynamic property ofthe corresponding atlas segment which is associated with a secondpatient class is determined. Depending on which degree of similarity ishigher, the type of the patient can be determined. For example, in casethe first degree of similarity is higher than the second degree ofsimilarity, it is determined that the patient is classified into thefirst class, i.e. the patient type is a patient type corresponding tothe first class.

The degree of similarity may be determined as described above withrespect to the comparing, i.e. a difference in dynamic characteristicssuch as values and/or a correlation between values and/or trajectoriesor else is used as a measure of a degree of similarity. Alternatively oradditionally, the aforementioned distribution may be used in thiscontext for the assessment of similarity. For example, the dynamicproperty of the patient segment is compared with the distribution of thedynamic property of the corresponding atlas segment which is associatedwith a first patient class. This may result in a first probability ofthe patient segment to fall into the first patient class. For example,the dynamic property of the patient segment is then compared with thedistribution of the dynamic property of the corresponding atlas segmentwhich is associated with a second patient class. This may result in asecond probability of the patient segment to fall into the secondpatient class. The first and the second probability may be used as ameasure of similarity.

The types of a patient may for example include a type of breathing ofthe patient, for example indicating an amount of diaphragmatic,thoracic, clavicular and/or paradoxical breathing. The types of apatient may for example include a type of a certain pathologicalappearance or a type of a certain disease.

The invention also relates to a use of the dynamic anatomic atlas formatching a patient image and an atlas image. This use for exampleincludes a step of using the information on the dynamic property of atleast one atlas segment as a constraint for the matching. For example,the matching includes an image fusion.

For example, a patient image (e.g. a static patient image comprisingonly static image information) may be matched with an atlas image. Inthis case, a constraint for the matching may be defined by the dynamicproperty of an atlas segment. For example, it may be defined that thecorresponding patient segment must have a physical property which lieswithin a certain range defined by the dynamic property of thecorresponding atlas segment and a predetermined range threshold. Forexample, the location of a possibly corresponding patient segment may beconstrained to a certain area. For example, the temperature ortemperature change of a possibly corresponding patient segment may beconstrained to a certain range. For example, the concentration of asubstance within the possibly corresponding patient segment may beconstrained to a certain range. In other words, static atlas data may beused for a matching, but in this example the dynamic atlas data is usedin addition to the static atlas data for the matching in the form of aconstraint, e.g. a constraint limiting the possible positions of acorresponding patient segment (corresponding to a corresponding atlassegment).

For example, a patient image comprising static and dynamic informationmay be matched with an atlas image. In this case, a constraint for thematching may be that a certain atlas structure to be matched with thepatient image exhibits a certain dynamic property which thecorresponding patient structure must also exhibit to a predetermineddegree or to which the corresponding patient structure must not be incontradiction thereto.

For instance the dynamic property may describe a maximal change ofgeometry and/or position of a segment in absolute terms or relative toother segments (e.g. maximum distance to another segment duringpositional change caused by vital movements), which should not beviolated by the image fusion. For example, in case the information onthe dynamic property respectively linked to an atlas structurerepresenting a rib describes a movement of max. 5 cm in z-direction aconstraint for the matching may be that the corresponding patientstructure is allowed to move max. 4-6 cm in z-direction which movementmay be described by the information on the dynamic property respectivelylinked to the corresponding patient segment.

Alternatively or additionally only atlas structures exhibiting a certaindegree of dynamic property such as a certain amount of movement are usedfor matching the patient image with an atlas image. For example, theatlas structure which exhibit the least amount of movement (indicated bythe information on the dynamic property respectively linked to the atlassegments) are used for the matching.

For example, the dynamic anatomic atlas is used for matching two patientimages (with one another). To this end, the matching is for examplecoupled (coupled transformation) by means of the atlas as described inthe chapter “Definitions”, in particular in the aspects 1) to 10). Forexample, the information on the dynamic property of at least one atlassegment is used as a constraint for the matching of the two patientimages. For example, the first patient image is an image representing acertain patient at a first point in time, whereas the second patientimage is an image representing the certain patient at a second point intime which is different from the first point in time. For example, thesecond point in time is later than the first point in time. For example,each of the first and the second patient images includes at least onepatient segment such as an anatomical segment (e.g. bladder). Forexample, at least one particular patient segment is included in both thefirst and the second patient image.

For example, a first patient image to be matched with a second patientimage is in a first step matched with a static atlas image generatedfrom the static atlas data of the dynamic anatomic atlas. For thismatching, for example an atlas segment which information on the dynamicproperty indicates a low rate of movement and/or a low change inposition, i.e. a dynamic property which lies below a predeterminedthreshold, is used (for example a vertebra which moves less than 1 mmaccording to the information on the dynamic property respectively linkedto the vertebra as atlas segment).

For example, the corresponding patient segments (e.g. lung, bladder,ribs) and the corresponding atlas segments (e.g. lung, bladder, ribs)are identified in a second step following the first step.

For example, the two patient images (the first patient image and thesecond patient image) are matched in a third step following the secondstep. For example, the information on the dynamic property (e.g.describing movement) respectively linked to one of the correspondingatlas segments (e.g. bladder) is used as a constraint for matching thefirst patient image with the second patient image. For example, it canbe determined that the positional difference between the correspondingpatient segment in the first patient image (e.g. bladder of the patientat the first point in time) and a patient segment identified in thesecond image has to lie in a certain interval, for example be lower thana maximal positional deviation described by the information on thedynamic property of the corresponding patient segment. In case thiscondition is not fulfilled, it can for example be determined that thepatient segment identified in the second patient image and used formatching the corresponding patient segment of the first patient image isnot the corresponding patient segment of the second patient image (e.g.in case the patient segment identified in the second patient image is akidney whereas the corresponding patient segment in the first patientimage is the bladder). In this case, another patient segment may beidentified in the second patient image to be matched with thecorresponding patient segment of the first patient image (e.g. thebladder). Alternatively or additionally, a second corresponding patientsegment (e.g. a kidney) of the first patient image may be determinedwhich fulfills the constraint for the matching with respect to thepatient segment of the second patient image (e.g. the kidney). Ofcourse, other kinds of dynamic properties can be used as constraint forthe matching instead of the maximal positional deviation, for examplemaximum and/or minimum temperature change, maximum deformation, upperand lower limit of a change in oxygen saturation or else.

The anatomic atlas can for example be used by acquiring the static atlasdata and the dynamic atlas data of the dynamic atlas, the static atlasdata describing a static atlas image of the atlas segments. In a nextstep, static patient data describing a static patient image of a patientsegment may be acquired. In a next step, the static patient image may bematched with the static atlas image. A corresponding atlas segmentcorresponding to the patient segment may be determined based on thematching.

The use of the dynamic anatomic atlas may further comprise a step ofdetermining subsegments within the patient segment based on the atlassubsegments of the corresponding atlas segment. For example, theposition and/or shape of the corresponding atlas subsegments(corresponding to the patient segment) are determined in coordinates ofthe static atlas image. These coordinates may be transformed into thecoordinates of the static patient image based on on the matching result,i.e. based on the (matching) transformation (matrix). Subsequently, theposition and/or shape of subsegments within the patient segment may bedetermined based on the transformed coordinates.

The dynamic property is a dynamic physical property like for example atleast one of dynamic spatial information or dynamic thermodynamicinformation or fluid-dynamic information. The dynamic spatialinformation for example comprises information on a change of position(movement) of an object (e.g. described by a trajectory) and/orinformation on a change of geometry (deformation) of an object. Thedynamic thermodynamic information for example comprises information on achange of temperature (e.g. heating-up or cooling-down) of an object,information on a change of pressure (e.g. pressure increase or pressuredecrease) of an object, information on a change of volume of an objecte.g. (expansion or contraction). The fluid-dynamic information comprisesfor instance information on a change of flux, velocity or density (e.g.concentration) of a substance within an object (e.g. change of oxygen inbrain vessel, change of flux of blood in (heart) arteries etc.). All ofthe aforementioned dynamic information may describe time-dependentphysical properties of an object. The object is at least one of thepatient segment or a subsegment thereof or one of the atlas segments ora subsegment thereof.

The invention also relates to a computer program which, when running onat least one processor of at least one computer or when loaded into thememory of at least one computer, causes the at least one computer toperform the aforementioned method, or a signal wave, for example adigital signal wave, carrying information which represents the program.It also relates to a non-transitory computer-readable program storagemedium on which the aforementioned program is stored. It also relates toleast one computer, comprising at least one processor and a memory,wherein the aforementioned program is running on the at least oneprocessor or is loaded into the memory, or wherein the at least onecomputer comprises the aforementioned program storage medium.

Definitions

In this section, definitions for specific terminology used in thisdisclosure are offered which also form part of the present disclosure.

A universal atlas can for example be generated by a data processingmethod for determining data which are referred to as atlas data andcomprise information on a description of an image of a generalanatomical structure, wherein this image is referred to as the atlasimage, the method comprising the following steps performed by acomputer:

-   -   acquiring patient data which comprise a description of a set of        images of an anatomical structure of a set of patients, wherein        the images are referred to as patient images and each patient        image is associated with a parameter set which comprises one or        more parameters given when the patient images are generated,        wherein the parameters influence representations of anatomical        elements as expressed by image values in the patient images, the        patient data comprising the patient image set and the parameter        sets associated with the patient image set;    -   acquiring model data which comprise information on a description        of an image of a model of an anatomical structure of a patient        which is referred to as the model image and is associated with        the parameter set;    -   wherein the model of an anatomical structure is referred to as        the model structure and comprises a model of at least one        anatomical element which is referred to as model element;    -   wherein the model data comprise:        -   model spatial information on a description of the spatial            information on the model structure; and        -   model element representation information on a description of            a plurality of representation data sets which contain            information on representations of the at least one model            element in the model images to be generated and are referred            to as model representation data sets, wherein the model            element representation information also describes a            determination rule for determining out of the plurality of            representation data sets (Table 3) respective model            representation data sets for one or more respective model            elements in accordance with respective parameter sets, the            representation data sets do not include spatial information            relating to the at least one model element;    -   wherein acquiring the model data involves generating, on the        basis of the model data and the patient data, the set of model        images which respectively represent at least a part of the model        structure by using the spatial information on the model        structure and particular model representation data sets which        are determined by applying the determination rule in accordance        with the one or more associated parameter sets and at least one        particular model element referred to as corresponding model        element, which is to be matched to at least one corresponding        anatomical element represented in the patient image and referred        to as patient element;    -   determining matching transformations which are referred to as PM        transformations and which are constituted to respectively match        the set of patient images of the set of patients to the set of        model images by matching images associated with the same        parameter set;    -   determining an inverse average transformation by applying an        inverting and averaging operation to the determined PM        transformations; and determining the atlas data by:        -   applying the determined inverse average transformation to            the model data; or        -   respectively applying the determined PM transformations to            the respective patient images in order to determine matched            patient images, averaging the matched patient images in            order to determine an average matched patient image, and            determining the atlas data by applying the determined            inverse average transformation to the average matched            patient image. One property of the universal atlas is for            example that the spatial information (e.g. positions and/or            geometry) and the representation information (e.g. grey            values) are stored separately. For further details on the            generation of the universal atlas, it is also referred to            PCT/EP2013/072005 published as WO2014/064063.

Matching by the universal atlas (e.g. using the universal atlas) can forexample be performed using the method according to one of the followingaspects 1) to 10). In particular, aspects 4) to 7) concern the matchingbetween several patient images of different modalities using theuniversal atlas.

Aspect 1) A data processing method for determining a matchingtransformation for matching a set of one or more images of an anatomicalbody structure of a patient, referred to as a patient image set, and aset of one or more images of a general anatomical structure, referred toas an atlas image set, wherein the general anatomical structurecomprises a plurality of anatomical elements referred to as atlaselements, and each patient image is associated with one of a pluralityof different parameter sets, wherein the parameter sets comprise one ormore parameters which obtain when the patient images are generated, andthe parameters influence representations of anatomical elements in thepatient images, the method comprising the following steps performed by acomputer:

-   -   acquiring atlas data, comprising the sub-steps of        -   acquiring atlas spatial information which contains spatial            information on the general anatomical structure, and        -   acquiring element representation information which describes            a plurality of representation data sets, wherein the element            representation information further describes a determination            rule for determining out of the plurality of representation            data sets respective representation data sets for respective            atlas elements in accordance with different respective            parameter sets, the representation data sets containing            information on representations of the plurality of atlas            elements in the atlas images to be generated but not            containing the spatial information on the general anatomical            structure;    -   acquiring patient data, comprising the sub-steps of        -   acquiring the patient image set, and        -   acquiring one or more of the plurality of parameter sets            which are respectively associated with the one or more            images of the patient image set;    -   generating, on the basis of the atlas data and the patient data,        the set of atlas images which respectively represent at least a        part of the general anatomical structure by using the spatial        information on the general anatomical structure and particular        representation data sets which are determined by applying the        determination rule in accordance with the one or more associated        parameter sets and particular atlas elements acquired and        referred to as corresponding elements, which are to be matched        to corresponding anatomical elements represented in the patient        image;    -   determining the matching transformation which matches the atlas        image set and the patient image set, by matching images        associated with the same parameter set to each other.

Aspect 2) The data processing method according to aspect 1), whereindetermining the atlas image set involves:

-   -   determining the representation data sets for the corresponding        elements, wherein for each atlas image to be determined, one of        the representation data sets is determined for each of the        corresponding elements in accordance with the determination        rule, wherein the determination rule comprises an assignment        rule for assigning a respective representation data set to a        respective corresponding element in accordance with the        parameter set associated with the patient image to which the        atlas image which includes the corresponding element is to be        matched; and    -   determining the atlas image set comprising one or more images        which are respectively associated with one of the parameter        sets, by respectively using the determined representation data        sets to determine the representations of the corresponding        elements.

Aspect 3) The data processing method according to any one of thepreceding aspects, wherein in order to determine the representation ofone or more of the corresponding elements in the one or more atlasimages, image values of patient elements are used in combination withdetermining the matching transformation.

Aspect 4) The data processing method according to any one of thepreceding aspects, wherein the step of determining the matchingtransformation, which matches one of the atlas images and one of thepatient images associated with one of the parameter sets to each other,is configured such that the matching transformation is determined on thebasis of information on the matching transformation between another ofthe atlas images and another of the patient images associated withanother of the associated parameter sets.

Aspect 5) The data processing method according to any one of thepreceding aspects, wherein the matching transformation is designed todeform a part of the geometry of the general anatomical structure inorder to match the atlas images to the patient images, and whereindetermining the matching transformation involves taking into accountinformation on the influence on matching quality of a deformation of atleast one of the atlas images associated with at least one of theparameter sets in order to determine the deformation of at least anotherof the atlas images which is associated with at least another of theparameter sets and includes corresponding elements which are identicalto the corresponding elements included in said at least one of the atlasimages.

Aspect 6) The data processing method according to the preceding aspect,wherein determining the matching transformation involves taking intoaccount the fact that the spatial information described by the atlasimages is identical and also taking into account information on thespatial correlation between the spatial information described by thepatient images in order to determine deformations described by thematching transformation which is applied in order to match the atlasimages and patient images to each other.

Aspect 7) The data processing method according to any one of thepreceding aspects, wherein the matching transformation comprises a setof coupled transformations referred to as matching sub-transformations,wherein the respective matching sub-transformations respectively matchthe atlas images associated with one of the associated parameter setsand the patient image which is associated with the same respectiveassociated parameter set to each other, and the matchingsub-transformations are coupled in that they each influence thedetermination of the other.

Aspect 8) The data processing method according to any one of thepreceding aspects, wherein the determination rule describes anassignment between the plurality of atlas elements and the plurality ofrepresentation data sets by describing a surjective assignment betweenthe atlas elements and representation classes, wherein the respectiverepresentation classes respectively represent subsets of the pluralityof representation data sets, and wherein for each of the respectiverepresentation classes, there is a unique set of characteristicbijective assignments between individual representation data sets of thesubsets and individual parameter sets.

Aspect 9) The data processing method according to any one of thepreceding aspects, wherein the representation data sets describe atleast one of the following types of information on representation: imagevalues for the anatomical elements; ranges of image values for theanatomical elements; the relationship between image values of differentanatomical elements; the relationship between image values for one ormore of the anatomical elements represented in images associated withdifferent parameter sets; maximum image values for the anatomicalelements; minimum image values for the anatomical elements; averageimage values for the anatomical elements; standard deviations of theaverage image values and structures of modulations of the image valuesfor the anatomical elements; characteristics of transitions betweenrepresentations of different anatomical elements.

Aspect 10) The data processing method according to any one of thepreceding aspects, wherein the atlas data also comprise spatialflexibility information which describes a flexibility of the position ofatlas elements within the general anatomical structure, and wherein thematching transformation is determined on the basis of the spatialflexibility information.

For further details on the aspects 1) to 10) relating to the matchingusing the universal atlas it is also referred to PCT/EP2012/071241published as WO2014/063746.

The method in accordance with the invention is for example a computerimplemented method. For example, all the steps or merely some of thesteps (i.e. less than the total number of steps) of the method inaccordance with the invention can be executed by a computer (forexample, at least one computer). An embodiment of the computerimplemented method is a use of the computer for performing a dataprocessing method. An embodiment of the computer implemented method is amethod concerning the operation of the computer such that the computeris operated to perform one, more or all steps of the method.

The computer for example comprises at least one processor and forexample at least one memory in order to (technically) process the data,for example electronically and/or optically. The processor being forexample made of a substance or composition which is a semiconductor, forexample at least partly n- and/or p-doped semiconductor, for example atleast one of II-, III-, IV-, V-, VI-semiconductor material, for example(doped) silicon and/or gallium arsenide. The calculating steps describedare for example performed by a computer.

Determining steps or calculating steps are for example steps ofdetermining data within the framework of the technical method, forexample within the framework of a program. A computer is for example anykind of data processing device, for example electronic data processingdevice. A computer can be a device which is generally thought of assuch, for example desktop PCs, notebooks, netbooks, etc., but can alsobe any programmable apparatus, such as for example a mobile phone or anembedded processor. A computer can for example comprise a system(network) of “sub-computers”, wherein each sub-computer represents acomputer in its own right. The term “computer” includes a cloudcomputer, for example a cloud server. The term “cloud computer” includesa cloud computer system which for example comprises a system of at leastone cloud computer and for example a plurality of operativelyinterconnected cloud computers such as a server farm. Such a cloudcomputer is preferably connected to a wide area network such as theworld wide web (WWW) and located in a so-called cloud of computers whichare all connected to the world wide web. Such an infrastructure is usedfor “cloud computing”, which describes computation, software, dataaccess and storage services which do not require the end user to knowthe physical location and/or configuration of the computer delivering aspecific service. For example, the term “cloud” is used in this respectas a metaphor for the Internet (world wide web). For example, the cloudprovides computing infrastructure as a service (IaaS). The cloudcomputer can function as a virtual host for an operating system and/ordata processing application which is used to execute the method of theinvention. The cloud computer is for example an elastic compute cloud(EC2) as provided by Amazon Web Services™. A computer for examplecomprises interfaces in order to receive or output data and/or performan analogue-to-digital conversion. The data are for example data whichrepresent physical properties and/or which are generated from technicalsignals. The technical signals are for example generated by means of(technical) detection devices (such as for example devices for detectingmarker devices) and/or (technical) analytical devices (such as forexample devices for performing (medical) imaging methods), wherein thetechnical signals are for example electrical or optical signals. Thetechnical signals for example represent the data received or outputtedby the computer. The computer is preferably operatively coupled to adisplay device which allows information outputted by the computer to bedisplayed, for example to a user. One example of a display device is anaugmented reality device (also referred to as augmented reality glasses)which can be used as “goggles” for navigating. A specific example ofsuch augmented reality glasses is Google Glass (a trademark of Google,Inc.). An augmented reality device can be used both to input informationinto the computer by user interaction and to display informationoutputted by the computer. Another example of a display device would bea standard computer monitor comprising for example a liquid crystaldisplay operatively coupled to the computer for receiving displaycontrol data from the computer for generating signals used to displayimage information content on the display device. A specific embodimentof such a computer monitor is a digital lightbox. The monitor may alsobe the monitor of a portable, for example handheld, device such as asmart phone or personal digital assistant or digital media player.

The expression “acquiring data” for example encompasses (within theframework of a computer implemented method) the scenario in which thedata are determined by the computer implemented method or program.Determining data for example encompasses measuring physical quantitiesand transforming the measured values into data, for example digitaldata, and/or computing the data by means of a computer and for examplewithin the framework of the method in accordance with the invention. Themeaning of “acquiring data” also for example encompasses the scenario inwhich the data are received or retrieved by the computer implementedmethod or program, for example from another program, a previous methodstep or a data storage medium, for example for further processing by thecomputer implemented method or program. Generation of the data to beacquired may but need not be part of the method in accordance with theinvention. The expression “acquiring data” can therefore also forexample mean waiting to receive data and/or receiving the data. Thereceived data can for example be inputted via an interface. Theexpression “acquiring data” can also mean that the computer implementedmethod or program performs steps in order to (actively) receive orretrieve the data from a data source, for instance a data storage medium(such as for example a ROM, RAM, database, hard drive, etc.), or via theinterface (for instance, from another computer or a network). The dataacquired by the disclosed method or device, respectively, may beacquired from a database located in a data storage device which isoperably to a computer for data transfer between the database and thecomputer, for example from the database to the computer. The computeracquires the data for use as an input for steps of determining data. Thedetermined data can be output again to the same or another database tobe stored for later use. The database or database used for implementingthe disclosed method can be located on network data storage device or anetwork server (for example, a cloud data storage device or a cloudserver) or a local data storage device (such as a mass storage deviceoperably connected to at least one computer executing the disclosedmethod). The data can be made “ready for use” by performing anadditional step before the acquiring step. In accordance with thisadditional step, the data are generated in order to be acquired. Thedata are for example detected or captured (for example by an analyticaldevice). Alternatively or additionally, the data are inputted inaccordance with the additional step, for instance via interfaces. Thedata generated can for example be inputted (for instance into thecomputer). In accordance with the additional step (which precedes theacquiring step), the data can also be provided by performing theadditional step of storing the data in a data storage medium (such asfor example a ROM, RAM, CD and/or hard drive), such that they are readyfor use within the framework of the method or program in accordance withthe invention. The step of “acquiring data” can therefore also involvecommanding a device to obtain and/or provide the data to be acquired. Inparticular, the acquiring step does not involve an invasive step whichwould represent a substantial physical interference with the body,requiring professional medical expertise to be carried out and entailinga substantial health risk even when carried out with the requiredprofessional care and expertise. In particular, the step of acquiringdata, for example determining data, does not involve a surgical step andin particular does not involve a step of treating a human or animal bodyusing surgery or therapy. In order to distinguish the different dataused by the present method, the data are denoted (i.e. referred to) as“XY data” and the like and are defined in terms of the information whichthey describe, which is then preferably referred to as “XY information”and the like.

Within the framework of the invention, computer program elements can beembodied by hardware and/or software (this includes firmware, residentsoftware, micro-code, etc.). Within the framework of the invention,computer program elements can take the form of a computer programproduct which can be embodied by a computer-usable, for examplecomputer-readable data storage medium comprising computer-usable, forexample computer-readable program instructions, “code” or a “computerprogram” embodied in said data storage medium for use on or inconnection with the instruction-executing system. Such a system can be acomputer; a computer can be a data processing device comprising meansfor executing the computer program elements and/or the program inaccordance with the invention, for example a data processing devicecomprising a digital processor (central processing unit or CPU) whichexecutes the computer program elements, and optionally a volatile memory(for example a random access memory or RAM) for storing data used forand/or produced by executing the computer program elements. Within theframework of the present invention, a computer-usable, for examplecomputer-readable data storage medium can be any data storage mediumwhich can include, store, communicate, propagate or transport theprogram for use on or in connection with the instruction-executingsystem, apparatus or device. The computer-usable, for examplecomputer-readable data storage medium can for example be, but is notlimited to, an electronic, magnetic, optical, electromagnetic, infraredor semiconductor system, apparatus or device or a medium of propagationsuch as for example the Internet. The computer-usable orcomputer-readable data storage medium could even for example be paper oranother suitable medium onto which the program is printed, since theprogram could be electronically captured, for example by opticallyscanning the paper or other suitable medium, and then compiled,interpreted or otherwise processed in a suitable manner. The datastorage medium is preferably a non-volatile data storage medium. Thecomputer program product and any software and/or hardware described hereform the various means for performing the functions of the invention inthe example embodiments. The computer and/or data processing device canfor example include a guidance information device which includes meansfor outputting guidance information. The guidance information can beoutputted, for example to a user, visually by a visual indicating means(for example, a monitor and/or a lamp) and/or acoustically by anacoustic indicating means (for example, a loudspeaker and/or a digitalspeech output device) and/or tactilely by a tactile indicating means(for example, a vibrating element or a vibration element incorporatedinto an instrument). For the purpose of this document, a computer is atechnical computer which for example comprises technical, for exampletangible components, for example mechanical and/or electroniccomponents. Any device mentioned as such in this document is a technicaland for example tangible device.

Preferably, atlas data is acquired which describes (for example defines,more particularly represents and/or is) a general three-dimensionalshape of the anatomical body part. The atlas data therefore representsan atlas of the anatomical body part. An atlas typically consists of aplurality of generic models of objects, wherein the generic models ofthe objects together form a complex structure. For example, the atlasconstitutes a statistical model of a patient's body (for example, a partof the body) which has been generated from anatomic information gatheredfrom a plurality of human bodies, for example from medical image datacontaining images of such human bodies. In principle, the atlas datatherefore represents the result of a statistical analysis of suchmedical image data for a plurality of human bodies. This result can beoutput as an image—the atlas data therefore contains or is comparable tomedical image data. Such a comparison can be carried out for example byapplying an image fusion algorithm which conducts an image fusionbetween the atlas data and the medical image data. The result of thecomparison can be a measure of similarity between the atlas data and themedical image data. The atlas data comprises positional informationwhich can be matched (for example by applying an elastic or rigid imagefusion algorithm) for example to positional information contained inmedical image data so as to for example compare the atlas data to themedical image data in order to determine the position of anatomicalstructures in the medical image data which correspond to anatomicalstructures defined by the atlas data. (emphasized part added by RB on 12Feb. 2016)

The human bodies, the anatomy of which serves as an input for generatingthe atlas data, advantageously share a common feature such as at leastone of gender, age, ethnicity, body measurements (e.g. size and/or mass)and pathologic state. The anatomic information describes for example theanatomy of the human bodies and is extracted for example from medicalimage information about the human bodies. The atlas of a femur, forexample, can comprise the head, the neck, the body, the greatertrochanter, the lesser trochanter and the lower extremity as objectswhich together make up the complete structure. The atlas of a brain, forexample, can comprise the telencephalon, the cerebellum, thediencephalon, the pons, the mesencephalon and the medulla as the objectswhich together make up the complex structure. One application of such anatlas is in the segmentation of medical images, in which the atlas ismatched to medical image data, and the image data are compared with thematched atlas in order to assign a point (a pixel or voxel) of the imagedata to an object of the matched atlas, thereby segmenting the imagedata into objects.

In the field of medicine, imaging methods (also called imagingmodalities and/or medical imaging modalities) are used to generate imagedata (for example, two-dimensional or three-dimensional image data) ofanatomical structures (such as soft tissues, bones, organs, etc.) of thehuman body. The term “medical imaging methods” is understood to mean(advantageously apparatus-based) imaging methods (for example so-calledmedical imaging modalities and/or radiological imaging methods) such asfor instance computed tomography (CT) and cone beam computed tomography(CBCT, such as volumetric CBCT), x-ray tomography, magnetic resonancetomography (MRT or MRI), conventional x-ray, sonography and/orultrasound examinations, and positron emission tomography. For example,the medical imaging methods are performed by the analytical devices.Examples for medical imaging modalities applied by medical imagingmethods are: X-ray radiography, magnetic resonance imaging, medicalultrasonography or ultrasound, endoscopy, elastography, tactile imaging,thermography, medical photography and nuclear medicine functionalimaging techniques as positron emission tomography (PET) andSingle-photon emission computed tomography (SPECT), as mentioned byWikipedia.

The image data thus generated is also termed “medical imaging data”.Analytical devices for example are used to generate the image data inapparatus-based imaging methods. The imaging methods are for exampleused for medical diagnostics, to analyze the anatomical body in order togenerate images which are described by the image data. The imagingmethods are also for example used to detect pathological changes in thehuman body. However, some of the changes in the anatomical structure,such as the pathological changes in the structures (tissue), may not bedetectable and for example may not be visible in the images generated bythe imaging methods. A tumor represents an example of a change in ananatomical structure. If the tumor grows, it may then be said torepresent an expanded anatomical structure. This expanded anatomicalstructure may not be detectable; for example, only a part of theexpanded anatomical structure may be detectable. Primary/high-gradebrain tumors are for example usually visible on MRI scans when contrastagents are used to infiltrate the tumor. MRI scans represent an exampleof an imaging method. In the case of MRI scans of such brain tumors, thesignal enhancement in the MRI images (due to the contrast agentsinfiltrating the tumor) is considered to represent the solid tumor mass.Thus, the tumor is detectable and for example discernible in the imagegenerated by the imaging method. In addition to these tumors, referredto as “enhancing” tumors, it is thought that approximately 10% of braintumors are not discernible on a scan and are for example not visible toa user looking at the images generated by the imaging method.

Image fusion can be elastic image fusion or rigid image fusion. In thecase of rigid image fusion, the relative position between the pixels ofa 2D image and/or voxels of a 3D image is fixed, while in the case ofelastic image fusion, the relative positions are allowed to change.

Elastic fusion transformations (for example, elastic image fusiontransformations) are for example designed to enable a seamlesstransition from one dataset (for example a first dataset such as forexample a first image) to another dataset (for example a second datasetsuch as for example a second image). The transformation is for exampledesigned such that one of the first and second datasets (images) isdeformed, for example in such a way that corresponding structures (forexample, corresponding image elements) are arranged at the same positionas in the other of the first and second images. The deformed(transformed) image which is transformed from one of the first andsecond images is for example as similar as possible to the other of thefirst and second images. Preferably, (numerical) optimization algorithmsare applied in order to find the transformation which results in anoptimum degree of similarity. The degree of similarity is preferablymeasured by way of a measure of similarity (also referred to in thefollowing as a “similarity measure”). The parameters of the optimizationalgorithm are for example vectors of a deformation field. These vectorsare determined by the optimization algorithm in such a way as to resultin an optimum degree of similarity. Thus, the optimum degree ofsimilarity represents a condition, for example a constraint, for theoptimization algorithm. The bases of the vectors lie for example atvoxel positions of one of the first and second images which is to betransformed, and the tips of the vectors lie at the corresponding voxelpositions in the transformed image. A plurality of these vectors ispreferably provided, for instance more than twenty or a hundred or athousand or ten thousand, etc. Preferably, there are (other) constraintson the transformation (deformation), for example in order to avoidpathological deformations (for instance, all the voxels being shifted tothe same position by the transformation). These constraints include forexample the constraint that the transformation is regular, which forexample means that a Jacobian determinant calculated from a matrix ofthe deformation field (for example, the vector field) is larger thanzero, and also the constraint that the transformed (deformed) image isnot self-intersecting and for example that the transformed (deformed)image does not comprise faults and/or ruptures. The constraints includefor example the constraint that if a regular grid is transformedsimultaneously with the image and in a corresponding manner, the grid isnot allowed to interfold at any of its locations. The optimizing problemis for example solved iteratively, for example by means of anoptimization algorithm which is for example a first-order optimizationalgorithm, such as a gradient descent algorithm. Other examples ofoptimization algorithms include optimization algorithms which do not usederivations, such as the downhill simplex algorithm, or algorithms whichuse higher-order derivatives such as Newton-like algorithms. Theoptimization algorithm preferably performs a local optimization. Ifthere is a plurality of local optima, global algorithms such assimulated annealing or generic algorithms can be used. In the case oflinear optimization problems, the simplex method can for instance beused.

In the steps of the optimization algorithms, the voxels are for exampleshifted by a magnitude in a direction such that the degree of similarityis increased. This magnitude is preferably less than a predefined limit,for instance less than one tenth or one hundredth or one thousandth ofthe diameter of the image, and for example about equal to or less thanthe distance between neighboring voxels. Large deformations can beimplemented, for example due to a high number of (iteration) steps.

The determined elastic fusion transformation can for example be used todetermine a degree of similarity (or similarity measure, see above)between the first and second datasets (first and second images). To thisend, the deviation between the elastic fusion transformation and anidentity transformation is determined. The degree of deviation can forinstance be calculated by determining the difference between thedeterminant of the elastic fusion transformation and the identitytransformation. The higher the deviation, the lower the similarity,hence the degree of deviation can be used to determine a measure ofsimilarity. A measure of similarity can for example be determined on thebasis of a determined correlation between the first and second datasets.

In particular, the invention does not involve or in particular compriseor encompass an invasive step which would represent a substantialphysical interference with the body requiring professional medicalexpertise to be carried out and entailing a substantial health risk evenwhen carried out with the required professional care and expertise. Forexample, the invention does not comprise a step of positioning a medicalimplant in order to fasten it to an anatomical structure or a step offastening the medical implant to the anatomical structure or a step ofpreparing the anatomical structure for having the medical implantfastened to it. More particularly, the invention does not involve or inparticular comprise or encompass any surgical or therapeutic activity.The invention is instead directed as applicable to positioning a toolrelative to the medical implant, which may be outside the patient'sbody. For this reason alone, no surgical or therapeutic activity and inparticular no surgical or therapeutic step is necessitated or implied bycarrying out the invention.

DESCRIPTION OF THE FIGURES

In the following, the invention is described with reference to theappended figures which represent a specific embodiment of the invention.The scope of the invention is however not limited to the specificfeatures disclosed in the context of the figures, wherein

FIG. 1 is a diagram showing the basic components of the discloseddynamic atlas;

FIG. 2 is a diagram showing an example of the information on the dynamicproperty;

FIG. 3 shows a first sequence of steps of a specific embodiment of thedisclosed method;

FIG. 4 shows a second sequence of steps of a specific embodiment of thedisclosed method;

FIG. 5 shows a principle configuration of a system of a specificembodiment of the invention.

FIG. 6 shows a flowchart related to the determination of trajectorysimilarity values as described in Annex A;

FIG. 7 shows a flowchart according to one exemplary embodiment accordingto at least one exemplary embodiment for determining dynamic DRRs asdescribed in Annex A;

FIG. 8 shows a flowchart according to one exemplary embodiment accordingto at least one exemplary embodiment for determining dynamic DRRs asdescribed in Annex A;

FIG. 9 shows a flowchart according to one exemplary embodiment accordingto at least one exemplary embodiment for determining dynamic DRRs asdescribed in Annex A;

FIG. 10 shows a flowchart according to one exemplary embodimentaccording to at least one exemplary embodiment for determining dynamicDRRs as described in Annex A;

FIG. 11 shows a flowchart according to one exemplary embodimentaccording to at least one exemplary embodiment for determining dynamicDRRs as described in Annex A;

FIG. 12 shows a schematic representation of a usual DRR which wasgenerated from a schematic planning CT in accordance with methods knownin the art as described in Annex A;

FIG. 13 shows a dynamic DRR generated from the same assumed schematicplanning CT according to an example as described in Annex A;

FIG. 14 shows a system according to at least one exemplary embodiment asdescribed in Annex A.

FIG. 1 is a diagram showing the basic components of the dynamic anatomicatlas 1. The dynamic anatomic atlas 1 comprises static atlas data 2 anddynamic atlas data 3. The static atlas data 2 describes a plurality ofatlas segments 4 a, 4 b, 4 c, 4 d whereas the dynamic atlas data 3comprises information on a dynamic property 5 a, 5 b, 5 c whichinformation is respectively linked to the atlas segments 4 a, 4 b, 4 c.In the example shown in FIG. 1, four different atlas segments 4 a, 4 b,4 c, 4 d are described by the static atlas data 2. The atlas segments 4a, 4 b, 4 c, 4 d in this example represent a first rib, a diaphragm, aheart, and a second rib. The information on the dynamic property 5 a, 5b, 5 c in this example is information on the movement of an anatomicalstructure described by a (closed loop) trajectory. The three differentinformation on the dynamic property 5 a, 5 b, 5 c shown in FIG. 1correspond to a trajectory of the atlas segment 4 a (e.g. the firstrib), a trajectory of the atlas segment 4 b (e.g. the diaphragm) and atrajectory of the atlas segment 4 c (e.g. the heart), each of which isassociated with the corresponding atlas segment 4 a, 4 b, 4 c in thedynamic anatomic atlas 1. In this example, the information on thedynamic property 5 a, 5 b, 5 c is stored as meta data associated withthe corresponding atlas segment 4 a, 4 b, 4 c. As noted in the generaldescription, other and/or additional dynamical (time-dependent) physicalproperties may be described by the information on the dynamic property.Also, correlations between the dynamic property of a first atlas segmentand the dynamic property of at least one other atlas segment may bestored as information on the dynamic property respectively linked to thefirst atlas segment.

In the example, there is no information on the dynamic propertyrespectively linked to atlas segment 4 d. This means that the dynamicanatomic atlas 1 shown in FIG. 1 comprises static atlas data 2describing atlas segments (namely 4 a, 4 b and 4 c) and dynamic atlasdata 3 comprising information on a dynamic property (namely 5 a, 5 b and5 c) which information is respectively linked to the atlas segments(namely 4 a, 4 b and 4 c).

FIG. 2 is a diagram showing an example of the information on the dynamicproperty. The example of FIG. 2 shows the information on the dynamicproperty 5 a described by the dynamic atlas data 3, wherein theinformation on the dynamic property 5 a is respectively linked to theatlas segment 4 a described by the static atlas data 2. In this example,the information on the dynamic property 5 a is represented as a matrix.Correlations between several different types of dynamical properties aredescribed by the information on the dynamic property, namely acorrelation of (movement) trajectories (first line in the matrix of FIG.2) and a correlation of temperature change (second line in the matrix ofFIG. 2).

The correlation in the first line, first column of the matrix is acorrelation between the trajectory of the atlas segment 4 a (e.g. thefirst rib) and the trajectory of the atlas segment 4 b (e.g. thediaphragm). The correlation in the first line, second column of thematrix is a correlation between the trajectory of the atlas segment 4 a(e.g. the first rib) and the trajectory of the atlas segment 4 c (e.g.the heart). The correlation in the first line, third column of thematrix is a correlation between the trajectory of the atlas segment 4 a(e.g. the first rib) and the trajectory of the atlas segment 4 d (e.g.the second rib). In the shown example, a numerical value (in theexample: 1, 9 and 5) as well as an indicator (in the example: “low”,“high” and “medium”) of the respective correlation is stored. Otherparameters may be stored for each of the correlations (e.g. a valueindicating correlation of the trajectories in a certain spatialdirection, a difference of maximum or minimum values of the trajectories(for example in a certain spatial direction) etc.), for example measuresof similarity of the trajectories.

The correlation in the second line, first column of the matrix is acorrelation between the temperature change of the atlas segment 4 a(e.g. the first rib) and the temperature change of the atlas segment 4 b(e.g. the diaphragm). The correlation in the second line, second columnof the matrix is a correlation between the temperature change of theatlas segment 4 a (e.g. the first rib) and the temperature change of theatlas segment 4 c (e.g. the heart). The correlation in the second line,third column of the matrix is a correlation between the temperaturechange of the atlas segment 4 a (e.g. the first rib) and the temperaturechange of the atlas segment 4 d (e.g. the second rib). In the shownexample, a numerical value (in the example: 97, 52 and 8) as well as anindicator (in the example: “high”, “medium” and “low”) of the respectivecorrelation is stored. Other parameters may be stored for each of thecorrelations (e.g. a value indicating correlation of the temperaturechanges in a certain time range, a value indicating correlation of therise in temperature, a value indication correlation of the sinking oftemperature etc.).

FIG. 3 shows a first sequence of steps of a specific embodiment of thedisclosed method. It should be noted that one or more of the methodsteps shown in FIGS. 3 and 4 can be performed at the same time orsubsequently, some of the steps may be replaced or omitted andadditional and/or alternative steps may be used, wherever possible. Inother words, the sequence of steps shown in FIGS. 3 and 4 is not theonly embodiment covered by this disclosure.

The method concerns the generation, improvement and/or enrichment of thedynamic anatomic atlas, in particular the generation of the informationon the dynamic property. The depicted method comprises several stepsS3.1 to S3.4. It should be noted that other and/or alternative and/oradditional steps can be used to generate the information on the dynamicproperty. For instance, several (f)MRT images may be used to determinethe change concentration of oxygen in certain segments.

In the first exemplary step S3.1, a 4DCT of at least one patient isacquired (e.g. loaded into a computer). Of course, other imagingmodalities are possible as long as image data is acquired in step S3.1which represents the patient at different points in time. For example, a4DCT scan can be used which includes several 3DCT scans and informationon their timely sequence (timely dependencies). For example, several3DCTs can be acquired which represent the patient at different points intime. These can be combined into a 4DCT.

In the next exemplary step S3.2, a group of voxels or each individualvoxel of a first 3D CT data set (e.g. acquired and/or generated in stepS3.1) is matched with a corresponding group of voxels or a correspondingindividual voxel of a second 3D CT data set which represents an image ofthe patient at a different point in time than the first 3D CT data set.Elastic fusion may be used for the matching. This step may be repeatedwith several image data representing the patient at further differentpoints in time. Consequently, the position of the group of voxels or ofthe individual voxel depending on the point in time can be determined.Connecting all the determined positions results in a (e.g. closed-loop)trajectory which describes the time-dependent movement of the group ofvoxel or of each individual voxel.

In the next exemplary step S3.3, movement correlation values arecalculated, for example for each individual voxel for which a trajectoryhas been determined (e.g. in step S3.2). The movement correlation valuesmay be determined by forming a correlation of a first trajectory of afirst voxel with a second trajectory of a second voxel. For example, aplurality of movement correlation values of a first trajectory of afirst voxel with respect to other trajectories of several (or all) othervoxels are determined.

In the next exemplary step S3.4, the trajectories are normalized. Forexample, the trajectories are normalized with respect to a referencetrajectory. For example, a first plurality of trajectories of a firstplurality of voxels are normalized in a different way (other reference,other normalization method, . . . ) than a second plurality oftrajectories of a second plurality of voxels. For example, all voxelswhich are part of a first anatomical body part are normalized in thesame manner. The anatomical body part may be determined by matching oneof the plurality of patient images with a static atlas image or by auser. For example, normalization is performed so that each patientsegment (representing an anatomical body part of the patient) isassociated with a certain trajectory and certain movement correlationvalues (e.g. by averaging the trajectories of all voxels within thepatient segment).

After normalization, the normalized trajectories and the movementcorrelation values (e.g. determined in step S3.3) which are associatedwith a certain patient segment are stored as dynamic atlas data 3 in thedynamic anatomic atlas 1. For this purpose, the normalized trajectoriesand the movement correlation values should be respectively linked to theindividual atlas segments. Therefore, at least one of the patient imagesused to obtain the normalized trajectories is matched with a staticatlas image. Image fusion may be used to determine a correspondingpatient segment which corresponds to a corresponding atlas segment.Afterwards, the information on the dynamic property (e.g. 5 a) of thecorresponding patient segment (the information for example comprisingthe normalized trajectory and the (normalized) movement correlationvalues, e.g. the trajectory of a rib) is stored in the dynamic anatomicatlas 1 respectively linked with the corresponding atlas segment (e.g.the atlas segment 4 a representing the rib). This results in the dynamicanatomic atlas 1 shown in FIG. 1.

FIG. 4 shows a second sequence of steps of a specific embodiment of thedisclosed method. For example, the method includes one or more of thesteps shown in FIG. 2 (S3.1 to S3.4) and continues with one of the stepsshown in FIG. 4. For example, the dynamic anatomic atlas 1 has alreadybeen created (is available), and the method starts with step S4.1. Inexemplary step S4.1, a 4DCT of a patient to be analyzed is acquired(e.g. loaded into a computer). As noted above, alternative imagingmodalities may be used as long as they include information on atime-dependent behavior of the patient, e.g. a series of patient imagesrepresenting the patient at different points in time.

In the next exemplary step S4.2, information of the dynamic propertiesis determined, i.e. trajectories are calculated for each or some of thevoxels of the patient image (e.g. acquired in step S4.3). The(closed-loop and/or cyclic) trajectories may be calculated as describedabove with respect to step S3.2 and may describe the time-dependentposition of a voxel or of a group of voxels.

In exemplary step S4.3, movement correlation values are calculated forevery voxel or for some of the voxels or for the groups of voxels. Asmovement correlation values, correlations between different trajectoriesmay be used (as described above with respect to step S3.3).

In a next exemplary step S4.4, correlation values between patienttrajectories and atlas trajectories are computed (e.g. determined orcalculated). For this purpose, only the trajectories are needed whichmeans that step S4.4 can directly follow step S4.3. For example, acorrelation between the trajectory of a corresponding patient segmentand the trajectory of a corresponding atlas segment is determined. Asnoted earlier with respect to FIG. 3, to identify the correspondingpatient and atlas segments, a patient image used to generate thetrajectories can be matched with a static atlas image (e.g. via imagefusion). Alternatively or additionally to step S4.5, step S4.5 may beperformed.

In exemplary step S4.5, the movement correlation values of the patientand of the atlas are compared. For example, the movement correlationvalues of a corresponding patient segment are compared with the movementcorrelation value of the corresponding atlas segment. The comparison mayinclude mathematical functions such as subtraction, division, addition,multiplication, differentiation, integration, a combination thereof orelse. As a result of the comparison, a certain numerical value may bedetermined.

Following step S4.4 and/or step S4.5, the correlation values determinedin step S4.4 and/or the comparison result determined in step S4.5 areinput into an analyzer in exemplary step S4.6. The analyzer uses one orboth of the aforementioned data to determine a degree of similaritybetween the patient and the atlas, for example between the dynamicproperty of the corresponding patient segment and the dynamic propertyof the corresponding atlas segment. This analysis may be used toclassify the patient according to a certain patient type. Alternativelyand/or additionally, this analysis may be used as an indicator for acertain disease (e.g. in case a certain patient segment moves differentcompared with a corresponding atlas segment, i.e. in case of a tumorwhich is attached to an anatomical structure which moves differently).Other ways of using the comparison between the dynamic property of oneor more patient segments and the dynamic property of the one or morecorresponding atlas segments are also possible, for example as laid outin the general description above.

FIG. 5 shows a principle configuration of a system of a specificembodiment of the invention: the system 6 comprises a computingenvironment 10 including at least one computer 7 having at least onedigital electronic processor which is operably coupled to at least oneelectronic data storage device 8 and an output device 9 (e.g. agraphical output device such as a display). The electronic data storagedevice 8 stores at least medical image data, the dynamic anatomic atlas1 or a program. The computer 7 is configured to output, to the outputdevice 9, electronic signals representing a (graphical) representationof a result of the data processing conducted by the computer 7.Furthermore, the computing environment 10 can be coupled to otherdevices such as a patient positioning device, a patient treatment device(for example a radiotherapy and/or radiosurgery device), one or moremedical imaging devices (imagers), other computing systems such as acloud based computing system or else.

According to an exemplary embodiment, based on elastically fused 4D CTsspecific points of a base CT can be transferred into CTs of differentbreathing phases, i.e. CTs of the patient describing the patient atdifferent points in time. The transferring results in differentpositions of the specific points in each of the images, which can berepresented by a (closed) trajectory in space. The target is now to findout, which trajectories are correlated in a specific patient, i.e. whichpoints behave similar under breathing. And for comparison it might beinteresting to know which points in a human body do normally correlate.A general correlation map is preferably generated in atlas space, storedas meta information of the Universal Atlas. But the averaging of metainformation is challenging since the information, which is used foraveraging, must be comparable between different patients independent oftheir breathing behavior. The information should therefore benormalized. The first step for an averaged correlation representation isto choose a set of points between which the correlation should becalculated. It can be done for every voxel or just for every organ orpart(s) of an organ. For this purpose, the human body, i.e. of theuniversal atlas, could be divided into a plurality of cells. A point(e.g. at the center) of each cell might thereafter be identified as areference for the individual cell. A number of 4D CTs of differentindividuals, which must be registered to the atlas (and or elasticallyfused to each other inside a 4D CT series) are required to create thedynamic atlas data. Then, the center points can be transformed (e.g.projected and/or matched) to the different data sets of the 4D CTseries. For each 4D CT series and each center point a trajectory can beobtained with points p_(i) (i=1 to n). An appropriate normalizedcorrelation measure is e.g. the cross correlation between twotrajectories p and q (or component-wise or weighted). All correlationsbetween all center point trajectories corr(cellk,celll) can becalculated as a huge matrix. This matrix can be averaged betweendifferent individuals and stored per cell pair as meta information inthe atlas. The direction of each trajectory can also be calculated. Thisdirection can also be averaged and stored per cell. The direction is nota scalar. It must be back-transformed into the atlas before averaging.

Annex A

One aspect of Annex A relates to the digital reconstructing (also called“rendering”) of three-dimensional x-ray images (CTs) intotwo-dimensional images. Those two-dimensional images are referred to asin the art as DRRs. The DRR represents a simulated two-dimensional x-rayunder the precondition of a particular (assumed) imaging geometry. Thedefinition of imaging geometry is given below. For example, therendering is performed so that the particular imaging geometrycorresponds to the imaging geometry of at least one (for example one ortwo) monitoring x-ray device (for generating two dimensional x-rayimages) which is used for monitoring a position of a patient in order toplace a patient for radiotherapy or radiosurgery in accordance with aplan (for example based on a planning CT). For example an isocenter ofthe radiotherapy or radiosurgery device and/or an isocenter of theplanning CT and/or an isocenter of the particular imaging geometryand/or and isocenter of the at least one monitoring x-ray device areidentical.

For example, in the medical field of radiotherapy or radiosurgery (inthe following, and in an unlimiting manner the term “radiotherapy” isused only, but has to be understood to cover at least one ofradiotherapy or radiosurgery), CTs are used for planning aradiotherapeutic treatment of a patient (for example to treat thetargets, for example tumors). The CTs used for planning aradiotherapeutic treatment are referred to in the art as “planning CTs”.Planning CTs are used to position the patient during theradiotherapeutic treatment. The radiotherapeutic treatment uses ionizingradiation (particles and/or electromagnetic waves) which are energeticenough to detach electrons from atoms or molecules inside the body andso ionize them. The treatment radiation is for example used inradiotherapy, for example in the field of oncology. For the treatment ofcancer in particular, the parts of the body comprising a tumor (which isan example for a “treatment body part”) are treated using the ionizingradiation. Since the body and in particular the treatment body part canbe moved during positioning of the patient for radiation treatment orduring the radiation treatment, it is advantageous to control theposition of the treatment beam such that the treatment beam hits thetreatment body parts as accurately as possible.

The movements of the treatment body parts are in particular due tomovements which are referred to in the following as “vital movements”.Reference is made in this respect to the European patent applications EP0 816 422 and EP 09 161 530 as well as EP 10 707 504 which discuss thesevital movements in detail.

In order to determine the position of the treatment body part,analytical devices such as x-ray devices, CT devices, and CBCT devicesare used to generate analytical images of the body. The analyticaldevices are in particular devices for analyzing a body of a patient, forinstance by using waves and/or radiation and/or beams of energy inparticular electromagnetic waves and/or radiation and/or ultrasoundwaves and/or particle beams. The analytical devices are in particulardevices which generate the above-mentioned two or three-dimensionalimages of the body of the patient (in particular of anatomical bodyparts) by analyzing the body.

However, it can be difficult to identify the treatment body part withinthe analytical image (for instance two-dimensional x-ray image). To thisend, the above-mentioned DRRs which are generated from a planning CT ina usual manner are used by an operator to identify the treatment bodypart in a two-dimensional x-ray image. To this end for instance the(usual) DRR is overlaid over an x-ray image generated when the patientis placed for treatment by means of the ionizing radiation or the DRR isplaced aside the two dimensional x-ray image on a display.

According to exemplary embodiments described in Annex A, there is atleast one “primary anatomical element”. This at least one primaryanatomical element corresponds for example to a treatment body part(e.g. tumor) or to one or more other anatomical elements (for examplesecondary anatomic elements). For example the one or more otheranatomical elements are anatomic elements which undergo a vitalmovement. For example, the other anatomical element is the heart,diaphragm, or rip cage or part thereof. For example, the at least oneprimary anatomic element is an anatomic element which is represented byat least one voxel (for example cluster of voxels) in for example theundynamic CT or planning CT. The at least one primary anatomical elementundergoes particular vital movements. The primary anatomical element canbe identified by an operator (for example physician or physicist) in anundynamic CT or in a planning CT. Other anatomical elements, inparticular the reminder of anatomical elements shown in the undynamic CTor the planning CT are referred to herein as secondary anatomicelements. Those secondary anatomical elements can or cannot undergovital movements or can or cannot undergo the same vital movements as theprimary anatomical elements. According to at least one exemplaryembodiment, an anatomical atlas is used for segmentation of theundynamic CT or the planning CT to identify at least one of primary andsecondary anatomical elements. According to at least one exemplaryembodiment, an anatomical atlas is used for segmentation of theundynamic CT or the planning CT to segments unlikely to undergo vitalmovements and to exclude those segments from a determination oftrajectories (see below) in order to save processing time and/or to makethe determination of the dynamic DRR more robust. For example avertebral column could be identified to be not subjected to vitalmovements and corresponding image elements of the 4D-CT could beexcluded from the determination of the trajectory similarity values asdescribed below.

According to an exemplary embodiment, the primary anatomical element isrepresented by at least one voxel, usually a cluster of voxels in theplanning CT. The term “a primary anatomical element” does not excludethat there is more than one anatomical element but covers the expression“at least one primary anatomical element”. If there is more than oneprimary anatomical element than those undergo the same vital movementsaccording to an exemplary embodiment. If there is more than one primaryanatomical element those are for example distinct, i.e. separated bysecondary anatomical elements. According to an exemplary embodiment,there are more than one primary anatomical element and for example themore than one primary anatomical elements are represented by a pluralityof imaging elements in the planning CT or 4D-CT. For example, at leastsome of which are adjacent. For example at least some of which aredistinct.

Acquisition of Basic Data

According to at least one exemplary embodiment, 4D-CT data (short“4D-CT”) are acquired. The 4D-CT represents a sequence ofthree-dimensional medical computer tomographic images (sequence of CTs)of an anatomical body part of a patient. The respectivethree-dimensional images (CTs) of the sequence for example represent theanatomical body part at different points in time. For example, theanatomical body part adopts different positions during a vital movement(e.g. caused by breathing and/or heartbeat). For instance, each CT (alsoreferred to as “volume” or “bin” in the art) corresponds to a specificrespiratory state which can be described as percentages of the fullyinhaled or fully exhaled state of the patient.

For example, a plurality of different respiratory states are describedby the sequence, for example, at least three, for example at least fivedifferent respiratory states are respectively described by at least oneCT (bin).

For example, the extremes of the cyclic movement (for instance maximuminhalation and/or maximum exhalation) are respectively described by oneCT of the sequence.

As mentioned above, one advantage of the exemplary embodiments describedherein is that additional information can be provided (for example to anoperator) which allows for a better interpretation and/or analysis ofthe CT and/or the two dimensional x-rays generated for monitoring theposition of the patient. According to at least one exemplary embodiment,one of the CTs (bins) of the sequence or a CT determined byinterpolation between two CTs defines the planning CT. For example, theinterpolation represents a state of the body part intermediate betweentwo neighboring states (respectively described by a sequence CT) whichare subsequently adopted by the body part which undergoes the vitalmovement (for example cyclic movement).

For example, if the 4D-CT does not define the planning CT (e.g. in thatone of the CT of the sequence is the planning CT or in that aninterpolation of at least two of the CTs of the sequence defines theplanning CT), then the planning CT is acquired separately.

Determination of Trajectory Similarity Values

In the following, the determination of trajectory similarity values isdescribed. This determination based on the 4D-CT represents in itself aseparate exemplary embodiment which can be supplemented by other stepsof other exemplary embodiments (for example a step of displaying thetrajectory similarity values) or the determination of the trajectorysimilarity values of image elements is embedded in at least oneexemplary embodiment as described herein.

According to at least one exemplary embodiment a three-dimensional imageis acquired from the 4D-CT. The acquisition of the image can forinstance be done by selecting one of the CTs (bins) of the sequencedefined by the 4D-CT or by determining a three-dimensional image bymeans of interpolation (as described above) from the 4D-CT. These threedimensional image is referred undynamic CT and for example comprises atleast one first image element representing the primary anatomicalelement. For instance, a plurality of voxels of the undynamic CTs (forinstance a cluster of voxels) represents the primary anatomical element(for instance target). For example, only one voxel represents aparticular one of the at least one primary anatomical element, forexample only one primary anatomical element. The second image elementsrepresent the secondary anatomical elements. For example the undynamicCT is selected by an operator from the sequence CTs to be that one inwhich a tumor is best discernable. An example for determining a CTsuitable for tumor identification and for positioning the patient isgiven in the following application: WO 2015/127970. According to atleast one exemplary embodiment, the undynamic CT is used to determinetrajectories. A trajectory which describes the path of a first imageelement and is referred to as “primary trajectory”. A primary trajectorydescribes the path of the first image element as a function of time. Forexample, the trajectory describes the path defined by positions of thefirst image element for different points in time which the first imageelement adopts in different sequence CTs. The different points in timecorrespond to different states of the cyclic movement (vital movement)of the primary anatomical element (for instance target). For example theprimary trajectory describes in a representative manner the trajectoryof more than one first image element as described below.

According to an exemplary embodiment, one of the first image elements inthe undynamic CT is defined to correspond to the isocenter of theplanning CT. For example, this first image element (which is for exampleone voxel or more voxels) is referred to as reference image element andused to determine a primary trajectory referred to as reference primarytrajectory which describes the path of the reference image element. forthis one image element. The reference primary trajectory can be used forcalculation of the trajectory similarity value as explained below.

According to a further exemplary embodiment, the reference image elementis defined to be that one which is the center of mass of the at leastone primary anatomical element (for example center of mass of tumor).Thus, the reference primary trajectory is the trajectory of the centerof mass. According to a further exemplary embodiment, the center of massand the isocenter are identical.

According to a further exemplary embodiment, the reference primarytrajectory can be acquired by determining a plurality of trajectorieseach one describing a trajectory of one or more of the at least onefirst image elements. Thus a plurality of trajectories are determinedwhich represent the movement of more than one first image element whichrepresent the at least one primary anatomical element. Then thereference primary trajectory is determined by averaging the plurality oftrajectories. The averaging can be performed by different mathematicalmethods, for instance by at least one of mean or mode or median or byweighing particular trajectories (for instance by weighing a trajectorywhich represents the center of the primary anatomical element (forinstance calculated by means of “center of mass” calculation where eachvoxel is assumed to have the same weight) or the isocenter of theplanned radiation treatment) or a combination of the aforementionedmethods.

The secondary trajectories respectively describe the trajectory of atleast one second image element. For example, a second trajectory maydescribe the trajectory of only one image element or the secondtrajectory may describe the trajectory of a plurality (e.g. cluster) ofsecond image elements. The determination of the first and second imageelements can in particular be performed by segmentation of the undynamicCT by using an anatomical atlas.

For example, image elements are excluded from trajectory determinationwhich are part of an anatomical segment (determined by means of anatlas) which is known to do not undergo vital movements.

According to an exemplary embodiment, the aforementioned at least oneprimary trajectory and the secondary trajectories are used fordetermining the trajectory similarity values. The trajectory similarityvalues respectively describe a similarity between the primary andsecondary trajectories. The trajectory similarity value describes inparticular a similarity in positional changes of the trajectories (forexample correlation, for example correlation coefficient) and/or asimilarity of amplitude of cyclic movement (for example similarity ofabsolute maximum and/or minimum amplitude of the cyclic movementdescribed by the compared trajectories).

According to at least one exemplary embodiment, a respective trajectorysimilarity value describes a similarity between a respective one of thesecond trajectories and one of the at least one primary trajectories(which is for example the reference primary trajectory) and/or between arespective one of the at least one primary trajectory and one of the atleast one primary trajectories (which is for example the referenceprimary trajectory).

The trajectory similarity value is for example calculated by using thesum of squared differences (or for example an absolute value function)for each coordinate in which the trajectories is described. The sum ofsquare of differences (or for example absolute value function) can beweighed in dependence on the coordinate. For example, the coordinatesystem is an orthogonal coordinate system. For example, one or more ofthe axes of the coordinate system are chosen to be directed along amajor movement direction of the vital movement, for exampleinferior-superior or anterior-posterior. For example, the axes of thecoordinate system are the main axes of a three dimensional surface (forexample surface of a rotational ellipsoid), the surface being spanned byat least one of the trajectories, for example the reference primarytrajectory which describes a cycling movement. For example, the mainaxes of the rotational ellipsoid can represent the axes of thecoordinate system. For example, one of the minuend and subtrahend of thesquared difference describes a deviation of a position one of the(primary or secondary) trajectory adopts at a particular point in time(that is the position of an image element (for example a first or secondimage element)) from an average position the trajectory adopts for theparticular point in time (the point in time being within the timecovered by the sequence described by the 4D-CT). For example, theaverage position is determined for one of the coordinate axes andaveraged over all points in time (of the sequence). For example, theother one of the minuend and subtrahend of the squared differencedescribes a position which is adopted by one of the primarytrajectories, for example by the reference primary trajectory. Thus, thesquared difference is a measure for deviation along an axis. Any otherfunction being a measure for such a deviation and the result of which isindependent from an algebraic sign, like the absolute value function,can be used.

The similarity values can also be calculated by using a calculation ofcorrelation coefficients which are for example a measure of thesimilarity of the trajectories.

The similarity measure (described by the trajectory similarity values)describes for example a similarity of the trajectories which describesfor example a similarity of the movement of the image elements describedby the trajectories.

The trajectory similarity values can be normalized. The trajectorysimilarity values can be a function of the peak to peak amplitude.According to exemplary embodiment, the trajectory similarity valuedescribes at least one of the following: the similarity of the movement(e.g. described by correlation coefficient or sum of square differences)or the similarity of the amplitude (for instance peak to peak amplitude)described by the trajectories or the frequency of the cyclic movementsdescribed by the trajectories. Details of examples of the calculation ofthe trajectory similarity value are given below in the description ofthe detailed exemplary embodiments. According to an exemplaryembodiment, the trajectory similarity value describes at least thecorrelation of the paths of the trajectories and/or of the movementsdescribed by the trajectories. According to an exemplary embodiment, foreach of the secondary trajectories, the trajectory similarity value iscalculated which describes for each of the secondary trajectories thecorrelation between the secondary trajectory and at least one of the atleast one primary trajectory, for example reference primary trajectory.According to an exemplary embodiment, the trajectory similarity valuedetermined in dependence on the correlation coefficient is additional afunction of the similarity of the amplitude and/or similarity of thefrequency. The function comprises in particular a threshold function.According to an exemplary embodiment, image values of a particular imageelement of the dynamic DRR are determined as a function of thetrajectory similarity values. For example image values are set to blacklevel (lowest brightness) during rendering of the DRR if all trajectorysimilarity values related to the image values of all image elements usedfor rendering the particular image element are lower than a thresholdvalue. According to another exemplary embodiment image values of imageelements of a planning CT are disregarded (for example by setting themto black level) during rendering of the dynamic DRR if the trajectorysimilarity value related to the image values of the image used forrendering (for example planning CT or dynamic planning CT) is lower thana threshold value or are changed in color value, for example set tolower brightness than before or changed in color, for example set to aparticular color (for example red). According to another exemplaryembodiment image elements of a dynamic planning CT are set to blacklevel if the trajectory similarity value related to them is lower than athreshold value or are changed in color value, for example set to lowerbrightness than before or changed in color, for example set to aparticular color (for example red). According to another exemplaryembodiment image values of the similarity image or the transformedsimilarity image are set to black level if the trajectory similarityvalue related to them is lower than a threshold value or are changed incolor value, for example set to lower brightness than before or changedin color, for example set to a particular color (for example red). Forexample, image values related to trajectory similarity values above apredetermined threshold remain unchanged are not influence by thetrajectory similarity values, and remain for example unchanged duringdetermination of the dynamic DRR or their color value is changed, forexample are set to higher brightness than before or changed in color(for example hue or saturation), for example set to a particular color(for example green), for example color different from that color set incase of below threshold value.

Determination of the Dynamic DRR

The trajectory similarity values determined as described above arepreferably used to determine the dynamic DRR. According to at least oneexemplary embodiment, the dynamic DRR is designed to reflect dynamicinformation on the movements (for example relative movement and/oramplitude and/or frequency) described by the at least one primarytrajectories (for example reference primary trajectory) and thesecondary trajectories, for example the movement relative to each other,the information being reflected in at least some of the image elementsof the dynamic DRR and reflect information of movement related to imageelements used for rendering the dynamic DRR. According to at least oneembodiment, the dynamic DRR reflects information on the dynamics ofanatomic elements in relationship to the dynamics of the at least oneprimary anatomic element. The information on dynamics (e.g. vitalmovement) is included in the dynamic DRR which is helpful foridentification of the at least one primary anatomic data elements (forexample helpful for more reliable target identification) in for example,the dynamic DRR and/or the dynamic CT and/or the similarity image. Theinformation on dynamics helps for an identification of secondaryanatomic elements having similar (for example same) vital movements asthe at least one primary anatomic element (for example target), inaddition to or alternatively to an identification of the at least oneprimary anatomic element. For example, those secondary anatomic elementsidentified in the dynamic DRR having similar (for example same) vitalmovement as the at least one primary anatomic elements are used forpositioning a patient (for example for radio therapeutic treatment) forexample relative to a beam arrangement (for example treatment beam).

If for example the least one primary anatomic element is an anatomicelement other than a treatment body part, like for example the heart ordiaphragm or rip cage or part thereof, the dynamic DRR and/or thedynamic CT and/or the similarity image allows to identify secondaryanatomic elements having similar (for example same) movement dynamics(for example undergo the same vital movements), for example move in thesame way as the heart or diaphragm or rip cage or part thereof.

According to at least one exemplary embodiment, the trajectorysimilarity values describe information on the dynamics, for examplemovements (for example relative movement and/or amplitude of (cyclic)movement and/or frequency of (cyclic) movement) described by the atleast one primary trajectories (for example reference primarytrajectory) and the secondary trajectories, for example information onthe dynamics, for example movement (for example relative movement and/oramplitude of (cyclic) movement and/or frequency of (cyclic) movement)relative to each other, for example information on the similarity of thedynamics (for example movements) described by the at least one primarytrajectories relative to the secondary trajectories.

If the 4D-CT does not define the planning CT but the planning CT isacquired independently, then preferably a transformation (referred to as“planning transformation”) from the undynamic CT to the planning CT isdetermined and used for determining the dynamic DRR. According to atleast one exemplary embodiment, at least a part of the image values ofthe image elements of the dynamic DRR is determined in dependence on thetrajectory similarity values. The dynamic DRRs can be calculated asknown in the art. That is, a particular imaging geometry can be defined.This imaging geometry is for instance defined by the position of anx-ray source and an x-ray detector. For instance, the imaginary rays ofthe x-ray source pass through a imaginary three-dimensional anatomicalbody part defined by the planning CT or the dynamic planning CT.According to at least one exemplary embodiment, the transmissionproperties of the image elements (for example voxels) are for exampledescribed by Hounsfield units and are for example defined by thebrightness of the respective voxels. According to at least one exemplaryembodiment, the trajectory similarity values assigned to the respectiveimage elements (e.g. voxels or clusters thereof) of thethree-dimensional image have an influence on the virtual absorptionproperties of the virtual three-dimensional anatomical body part withrespect to the virtual rays passing there through. According to otherexemplary embodiments, the image values of the respective image elements(e.g. voxels or clusters thereof) describing the virtualthree-dimensional anatomical body part and defining the absorptionproperties of the respective image elements (e.g. voxels or clustersthereof) are changed in dependence on the trajectory similarity valuesassigned to the respective voxels before the virtual rays pass throughthe virtual three-dimensional anatomic body part in order to determinethe dynamic DRR.

According to an aspect, the planning CT is not used for determining thedynamic DRR, and/or the similarity image and/or the dynamic CT. Forexample only the 4D-CT is used for determining the dynamic DRR and/orthe similarity image and/or the dynamic CT, this is for example done inorder to reflect the dynamics, in a static two or three imagesdimensional image or a sequence of those images, for example to getdeeper insight in the vital movements.

According to at least one exemplary embodiment, the image values ofimage elements of the dynamic DRRs are determined by using (for exampleconsidering) the trajectory similarity values such that the brightnessof the at least some of the image values are different compared to a DRRdetermined from the planning CT in a usual manner (i.e. not using thetrajectory similarity values, but anything else used for thedetermination, for example the assumed imaging geometry is the same),such a DRR being referred to herein as “usual DRR”. For example, theimage values being different relate to image elements representingsecondary anatomical elements. According to at least one exemplaryembodiment, the image values (for instance brightness) are changedcompared to the usual DRR as a function of the trajectory similarityvalues related to the secondary anatomical element represented by theimage value. Trajectory similarity values related to primary anatomicalelements are referred to herein as first trajectory similarity values.For example, the first trajectory similarity values are 1. Trajectorysimilarity values related to secondary anatomical elements are referredto herein as second trajectory similarity values and are for exampleequal to or lower than the first trajectory similarity values.

The term “related” mentioned above means for example, that they relateto the same particular anatomical element represented in at least onethree-dimensional matrix which describes at least one three dimensionalimage. For example, a trajectory similarity value is related (forexample assigned) to a particular image element (for instance voxel) ofthe planning CT (which particular image element has a particularposition in a matrix which describes the planning CT). For example animage value of a particular image element (e.g. voxel or clustersthereof) has been modified based on the trajectory similarity valuerelated to the particular image element, the particular image elementrepresenting a particular anatomical element.

Herein, the “positions” in a matrix mean that they relate to aparticular anatomical element represented by an image element (forexample voxel or cluster thereof) in a three dimensional image. “Samepositions” means that they relate to the same particular anatomicalelement.

Instead of setting image values of image elements (voxels) representingthe virtual three-dimensional anatomical body part to black level, it isalso possible to disregard those image elements (voxels) when virtuallypassing the rays there through during rendering of the dynamic DRR. Thatis, those image elements are handled as if no absorption of the virtualray happens at the location of the image element (for instance voxel).Correspondingly, if the image value (for instance brightness) is onlymodified and not set to for instance to minimum brightness (blacklevel), a corresponding procedure would be to modify correspondingly theabsorption of the virtual ray when passing to the corresponding imageelement (for instance voxel). As explained above, there are differentways to determine the dynamic DRR based on the determined trajectorysimilarity values. At least some of which will be explained below.

According to an exemplary embodiment, the undynamic CT is the planningCT. That is, the planning CT and the acquired undynamic CT areidentical. In this case, the step of determining the dynamic DRR uses,according to an exemplary embodiment, the planning CT and the determinedtrajectory similarity values for determining the dynamic DRR. Accordingto an exemplary embodiment, during determination of the DRR (for exampleduring rendering the DRR) from the planning CT, the trajectorysimilarity values are considered. According to an exemplary embodiment,the “consideration of the trajectory similarity values”, is performedwhen virtually passing the rays from the virtual radiation sourcethrough the virtual three-dimensional anatomical body part described bythe planning CT. For example, the image values describe the transmissionand/or absorption properties of the virtual three-dimensional bodyparts, for example by means of Hounsfield values (for example Hounsfieldunits). According to an exemplary embodiment, the transmission and/orabsorption properties described by the image values of the planning CTare modified in accordance with the trajectory similarity values relatedto (for example assigned to) the different positions of the threedimensional matrix representing the planning CT. For example, if atrajectory similarity value assigned to a particular position of thematrix indicates no similarity, then unattenuated transmission isdefined for the position during rendering of the dynamic DRR.

Herein, a change, for example a modification of an image value covers atleast one of change of brightness or change of color (for example changeof hue and/or change of saturation).

According to a further exemplary embodiment, the brightness values ofthe planning CT describes the transmission and/or absorption propertiesof anatomical elements represented by image values of the planning CT.For example, the brightness values are modified in accordance with thetrajectory similarity values assigned to the respective positions of thematrix describing the planning CT. Alternatively or additionally, thecolors of the image elements are modified in accordance with thetrajectory similarity values (for example red in case of low similarityand green in case of high similarity). According to this exemplaryembodiment, the planning CT is modified based on the trajectorysimilarity values assigned to the respective image elements (e.g.voxels) of the planning CT. That is, a modified planning CT isdetermined based on the trajectory similarity values. This modifiedplanning CT describes a modified virtual anatomical body part throughwhich the virtual rays pass in order to determine the dynamic DRR. Forexample elements of the virtual anatomical body part are fullytransmissive for x-ray, if trajectory similarity values related to theseelements are below a threshold value. The planning CT modified by thetrajectory similarity values respectively assigned to the image elementsof the planning CT is also referred to herein as “dynamic planning CT”.For example, the dynamic planning CT describes the transmission and/orabsorption properties of a virtual anatomical body part through whichthe virtual ray pass during rendering of the dynamic DRR. Sometimes inthe art, a CT generated by using contrast agents is referred to as a“dynamic CT”. Herein “dynamic” is used in a different manner and a“dynamic CT” or a “dynamic planning CT” can be generated by using acontrast agent or by not using a contrast agent. Correspondingly,“undynamic” is used in a different manner and a “undynamic CT” can begenerated by using a contrast agent or by not using a contrast agent.

According to further exemplary embodiments, the planning CT is notdetermined based on the 4D-CT but determined separately. According to anexemplary embodiment, in this case, a transformation is determined fromthe acquired undynamic CT to the planning CT.

Based on the trajectory similarity values determined as mentioned above,a three-dimensional image is acquired. This three-dimensional image isreferred to as “similarity image”. The positions of the image elements(for example voxels or clusters thereof) of the similarity image in amatrix which describes the similarity image correspond to positions ofimage elements of a matrix which describes the undynamic CT and theimage values of the image elements of the similarity image correspond tothe trajectory similarity values assigned to the corresponding imageelements of the undynamic CT. For example, “corresponding positions”means that the respective trajectory similarity values are at the samepositions in a matrix which describes the similarity image as the imageelements of another matrix which describes the undynamic CT to whichthey are respectively assigned.

For example, the transformation is applied to the similarity image inorder to determine a transformed similarity image. The transformedsimilarity image is transformed so that the image elements of thetransformed similarity image are at positions in a matrix whichdescribes the transformed similarity image which correspond to positionsof image elements of another matrix which describes the planning CT, thecorresponding positions relate to the same anatomical element. That is,the transformation results in that trajectory similarity values areassigned to the respective image elements of the planning CT.

For example, the dynamic DRR is determined by using the planning CT andthe determined trajectory similarity values wherein, duringdetermination of the DRR from the planning CT, the trajectory similarityvalues represented by the image elements of the transformed similarityimage are used. That is, the attenuation of the virtual ray passingthrough the virtual three-dimensional body represented by the planningCT is modified in dependence on the image values of the transformedsimilarity image being assigned to respective image elements of theplaying CT (as mentioned before). According to a further example, theimage elements of the planning CT are modified based on the transformedsimilarity image. As mentioned above, the transformed similarity imageallows to assign to each image element of the planning CT a trajectorysimilarity value which is a corresponding image value of the transformedsimilarity image. The assigned trajectory similarity value is used tochange the image values of the planning CT. The term “corresponding”means in this respect that the trajectory similarity values of thetransformed similarity image adopt the same position in the transformedsimilarity image as the corresponding image elements of the planning CTdo.

The planning CT modified as mentioned above is referred to herein as“dynamic planning CT”. The procedure for determining the DRR is appliedto the dynamic planning CT in order to determine the dynamic DRR.

According to at least one further exemplary embodiment, the planning CTis acquired independently from the undynamic CT as described above. Inthis case, for example, a transformation from the undynamic CT to theplanning CT is determined.

Furthermore, for example, a three-dimensional image (referred to asdynamic CT) is determined by changing image values of at least a part ofthe second image elements of the undynamic CT. The change of the imagevalues is performed in dependence on the trajectory similarity valuesassigned to respective image elements of the undynamic CT. In otherwords, for the respective image elements of the undynamic CT, therespectively assigned trajectory similarity values modify the respectiveimage value of the respective image element of the undynamic CT. Forexample, the trajectory similarity values are determined as mentionedabove for the respective image elements of the undynamic CT and thenassigned to the respective image elements of the undynamic CT for whichthey have been determined.

For example, the determined transformation is applied to the dynamic CTin order to determine a CT referred to as “dynamic planning CT”. That isthe transformation (transformations herein are spatial transformations)transforms the dynamic CT into the dynamic planning CT. At least a partof the second image elements of the dynamic planning CT reflect thepreviously determined correlation.

For determining the dynamic DRR, for example, the dynamic planning CT isused as a basis for digitally reconstructing the two-dimensional imagefrom the dynamic planning CT. That is, the virtual rays pass through avirtual anatomical body part, the transmission and/or absorptionproperties of the elements of the body part being described by the imagevalues of the dynamic planning CT.

According to an example of at least one exemplary embodiment, theprimary and secondary trajectories are determined as described in thefollowing. Transformations referred to as sequence transformations aredetermined. The sequence transformation describe transformations betweensequence CTs. For example a transformation from the undynamic CT toanother one of the sequence CTs (in case the undynamic CT is one of thesequence CTs). For example, the sequence transformations allow totransform between subsequent ones of the sequence CTs. For example, thesequence transformation are constituted to transform from the undynamicCT to other ones of the sequence CTs. The transformations are preferablyperformed by using image fusion. For example, the sequencetransformations are constituted so that the positions of the imageelements of a respective one of the sequence CTs can be transformed tothe positions of the respective image elements in another respective oneof the sequence CTs. Thus, the determined sequence transformations allowto determine a change of position of image elements in the sequence.This change of positions represents trajectories of anatomical elementsdescribed by the respective image elements.

For example, the trajectories of the at least one first image elementand of at least some of the second image elements are determined byapplying the determined sequence transformations to the at least onefirst image element and to the at least some of the second imageelements.

According to at least one exemplary embodiment, the trajectorysimilarity values are determined based on the trajectories. According toan example of the at least one exemplary embodiment, the trajectorysimilarity values are determined as a function which has a positivevalue and is the higher the higher an absolute value of a differencebetween a minuend and a subtrahend is. The function is referred to asabsolute difference function and is for example the function of squareddifferences, difference to the fourth power, sixth power . . . or afunction for obtaining an absolute value of the difference. The minuendand subtrahend depend on positions of two different trajectories at aparticular (same) time. One of the two trajectories being a primarytrajectory, according to an embodiment the reference primary trajectory.

For example the calculation of the trajectory similarity values can beperformed for each coordinate of a coordinate system in which thetrajectories are at rest. For instance, a first deviation (difference)of a first image element from a mean average value of the position ofthe first image element can be subtracted from a second deviation(difference) of a second image element from an average position withrespect to the same coordinate and then those two deviations aresubtracted and for example the absolute difference function is appliedto this difference.

The aforementioned positive values can be weighed differently for eachcoordinate axis in order to determine a value which reflects thecorrelation for example for all three axes of the coordination system.This determined value is for example the trajectory similarity value.Furthermore, a threshold function can be applied to value in order toobtain the trajectory similarity value.

According to at least one further exemplary embodiment, the trajectorysimilarity value is determined based on calculation of a correlationcoefficient. For example, the trajectory similarity value is a functionof a product of the aforementioned first and second deviations. Forexample, this function is calculated for each axis of the coordinationsystem. The different values for different axes of the coordinationsystem can be weighed. Optionally a threshold function can be applied tothe result of the function in order to obtain trajectory similarityvalues.

According to a further exemplary embodiment, the trajectory similarityvalue is a value referred to as amplitude similarity value. For example,the trajectory similarity value is a function, for example thresholdfunction of the amplitude similarity value. For example, the amplitudesimilarity value reflects similarity of amplitudes of first and secondimage elements while they undergo a cyclic (for instance periodic)movement. More details are given below in the detailed exemplaryembodiments. The aforementioned exemplary embodiments and examples fordetermining the trajectory similarity value can be combined. Accordingto a further exemplary embodiment both the correlation coefficient andthe amplitude similarity value (which describes for example similarityof a peak to peak amplitude) can be combined. For example, both thecorrelation coefficient and the amplitude similarity value arerespectively subjected to a threshold function having respectivethreshold values. For example, the trajectory similarity value isdetermined by using a function which sets the trajectory similarityvalue to a value which indicates similarity if both the correlationcoefficient and the amplitude similarity value are above theirrespective threshold values. If one of them is below, then thetrajectory similarity value is set to indicate “not similar” (which forexample results in that a corresponding image element in the dynamic DRRis set to black level).

According to at least one exemplary embodiment of Annex A, the computerimplemented method further comprises the steps of determining at leastone of the at least one first image element or the second image elementsby using an anatomical atlas. The steps in particular comprisesegmenting the undynamic CT by using the atlas. The segments achieved bymeans of the segmenting being identified to correspond to one or more(for instance clusters) of the second image elements and/or the at leastone first image element. In particular, image elements can be excludedfrom the processing (for example by not calculating the trajectories forthem) which are part of segments known to be not subjected to a vitalmovement or a vital movement which is not similar to that of thetreatment body part. Or for those image elements the trajectorysimilarity values are set to indicate no similarity.

According to at least one further exemplary embodiment, the computerimplemented method comprises the step of displaying the dynamic DRR overan x-ray image (for example by superposition) or besides an x-ray image.The x-ray image is for example used by an operator (for instance surgeonor physicist) to determine the position of a treatment body part to besubjected to treatment radiation. The display of the dynamic DRR can beused for (planning) the positioning of the patient for theradiotherapeutic treatment.

According to an example, image values (for example of the similarityimage) representing the trajectory similarity values can have abrightness or color (for example hue and/or saturation) which depends onthe trajectory similarity value.

According to a further aspect, a computer implemented method is providedwhich is for example used to determine the above mentioned similarityimage and/or dynamic CT and/or dynamic DRR. The determination is forexample based on a 4D-CT, for example not based on a planning CT, forexample uses (only) the 4D-CT. The 4D-CT describes for example asequence of three-dimensional medical computer tomographic images of ananatomical body part (referred to as sequence CTs). The sequence CTsrepresent the anatomical body part at different points in time. Theanatomical body part comprises at least one primary anatomical elementand secondary anatomical elements. This further aspect is for exampleused if no radiotherapeutic treatment is intended for the patient and ifthere is no need for a planning CT. This further aspect is for exampleused if further insights in the (anatomical) dynamics of the patient isrequired. With exception of the use of the planning CT, the methodaccording the further aspect comprises one or more step combinations asdescribed above. According to a further aspect, a complete implementmethod is provided which uses at least or only the steps shown in FIG. 1for determining the trajectory similarity values. According to a furtheraspect, a computer implemented method is provided that uses the stepsS20 and S24 of FIG. 2, while in step S24 the dynamic DRR is determinedby using the undynamic CT instead of the planning CT. According tofurther aspects, method uses the steps in FIG. 3 with exception of stepas 32. Furthermore, step 34 has changed in that the dynamic CT isdetermined by using the undynamic CT and the determined trajectorysimilarity values and by changing image values of the undynamic CTindependence on the trajectory similarity values. Finally the step S36has changed in that the dynamic DRR is determined from the dynamic CT.According to aspects, at least one of the dynamic DRR or the dynamic CTis displayed. According to a further aspect, the steps of FIG. 1 aresupplemented by step of displaying the determined trajectory similarityvalues as three-dimensional similarity image.

The computer implemented method according to the further aspectcomprises steps as mentioned below, examples for at least some of thesteps are described with respect to other aspects described herein andare therefore not (again) described in detail.

For example, the 4D-CT is acquired. A planning CT is acquired. Theplanning CT is according to a first exemplary embodiment acquired basedon the 4D-CT. For example, by interpolation between one of the sequencesCTs or by defining one of the sequence CTs to be the planning CT.According to a further alternative exemplary embodiment, the planning CTis acquired independently from the 4D-CT for example by receiving CTdata from a medical analytical imaging device which is constituted togenerate CTs.

For example the computer implemented method further comprises the stepof acquiring a three-dimensional image, referred to as undynamic CT,from the 4D-CT. For example, one of the sequence CTs is selected as theundynamic CT. The selection is for instance performed on a visual basis.For instance, an operator selects one of the sequence CTs in which atreatment body part can be visually best segmented from other bodyparts. According to a further example, a segmentation of the treatmentbody part by using an atlas has highest confidence level for thetreatment body part in case of the selected sequence CT. Theaforementioned features can be combined also with the other aspectsmentioned before.

In a further step, for example, a trajectory is acquired, the trajectoryis referred to as primary trajectory. The acquisition is for examplebased on the 4D-CT. The primary trajectory describes a path of the atleast one first image element as a function of time.

For example, in a further step, trajectories of the second imageelements are acquired. The trajectories are referred to as secondarytrajectories. The acquisition is for example based on the 4D-CT.

For example, in a step trajectory similarity values are determined. Thetrajectory similarity values are determined for the image values of theundynamic CT. The determination is for example based on the primarytrajectory and the secondary trajectories. The trajectory similarityvalues respectively describe a means for similarity as described herein.

For example, in another step, the similarity image is determined bydetermining the trajectory similarity values to be image values of imageelements of a similarity image. The image elements of the similarityimage are referred to as similarity image elements. The image elementsof the undynamic CT are referred to as undynamic image elements. Asdescribed with respect to other aspects, the determination of thesimilarity image is performed so that the positions of the similarityimage elements correspond to the positions of the undynamic imageelements of the undynamic CT to which the trajectory similarity valuesare respectively related.

The acquisition of a planning CT is optional. For example, thesimilarity image can be determined without using the planning CT.

Optionally, in case the planning CT is not acquired based on the 4D-CTbut independently from the 4D-CT, a transformation is further determinedfrom the undynamic CT to the planning CT (examples therefore aredescribed above with respect to the other aspect). For example thedetermined transformation is applied to the similarity image (examplestherefore are described herein with respect to the other aspects).

According to a further exemplary step, the similarity image or thetransformed similarity image is displayed For Example, the similarityimage is determined for each CT of the sequence CT. For example a changeof the similarity images is visualized by a movie play feature.

According to another exemplary embodiment of this aspect, the similarityimage or the transformed similarity image is displayed over or besides aCT, for example sequence CT and/or planning CT. According to anotherexemplary embodiment, a DRR (referred to as similarity DRR) is renderedusing the similarity image as the tree dimensional image in the mannerdescribed above. For example, the same imaging geometry is used for therendering of the similarity DRR as for generation of a two-dimensionalx-ray image which is for example used for placing a patient. Thesimilarity DRR is for example display over the two-dimensional x-ray(for example superposed) or displayed besides the two-dimensional x-rayimage.

According to a further aspect, a program is provided which when runningon a computer or when loaded into a computer causes the computer toperform at least one of the computer implemented methods describedherein.

According to a further aspect, a signal wave is provided, which carriesinformation which represent the program according to the aforementionedaspect.

According to a further aspect, a program is provided, which comprisescode means adapted to perform all the steps of at least one of thecomputer implemented methods described herein.

According to a further aspect of Annex A, a program storage medium isprovided, on which the program according to at least one of theaforementioned aspects is stored. The program is for example stored in anon-transitory manner.

According to a further aspect of Annex A, a computer is provided, onwhich the program according to at least one of the aforementionedaspects is running or in which such a program is loaded. The computer isfor example constituted to perform at least one of the aforementionedcomputer implemented methods. For example, the computer comprises theprogram storage medium of one of the aforementioned aspects.

According to further aspects of Annex A, a system is provided. Thesystem comprises for example the computer according to theaforementioned aspect. For example, the system further comprises adisplay device (for example a computer monitor) for displaying thedynamic DRR determined in accordance with one of the aforementionedaspects. For example, the display device is alternatively oradditionally constituted to display the similarity image according toone of the aforementioned aspects. For example, the computer comprisesan interface (for example a digital and/or electronic interface) forreceiving data, for example the 4D-CT and/or the planning CT.

According to a further exemplary embodiment of this aspect, the systemcomprises a couch for placing a patient, for example for treatment withtreatment radiation. The system for example further comprises accordingto this exemplary embodiment, a treatment device constituted to emit atreatment beam for treating the patient by means of treatment radiation.

According to a further exemplary embodiment of this aspect, the systemcomprises an analytical device constituted for generating the 4D-CT.

For example, according to a further exemplary embodiment, the systemalternatively or additionally comprises an analytical device constitutedfor generating the planning CT.

Description of FIGS. 6 to 14

FIG. 6 shows steps for determining the trajectory similarity values.According to step S12, the undynamic CT is acquired. According to stepS14, the primary and secondary trajectories are acquired. For example,the primary and secondary trajectories are determined based on theacquired undynamic CT, for example based on the at least one first imageelement and the second image elements. For example, the first imageelement is a tumor. For example the second image elements representsecondary anatomical elements. For example the secondary anatomicalelements are discernible in an x-ray image. For example, those secondaryanatomical elements have a strong interaction with x-rays (for exampleby absorbing the x-rays) than fluids (for example water, air).

Having a reference to FIG. 12, it is assumed that FIG. 12 represents aschematic usual DRR generated from the dynamic CT which is assumed tocorrespond to the planning CT. Then according to an example, region 1810represents the treatment body part and is generated from a cluster ofvoxels of the planning CT which corresponds to the undynamic CT. Thatis, the region 1810 in FIG. 12 corresponds to a cluster of first imageelements of the undynamic CT from which the usual DRR of FIG. 12 isgenerated. Accordingly, according to an example, the regions 1812, 1822,1824, 1826, and 1828 are generated from clusters of second imageelements of the undynamic CT (which is identical to the planning CT).

According to step S14 of the FIG. 6, primary and secondary trajectoriesare acquired based on first and second image elements of the undynamicCT and based on the other sequence CTs defined by the 4D-CT. Asmentioned above, preferably image fusion methods are used to determinethe trajectories. In a next step, for example, the trajectory similarityvalues related to the respective image elements of the undynamic CT aredetermined. For example, this is done for each voxel of the undynamic CTor for voxel clusters. According to an example, the trajectorysimilarity values for the voxels being inside the region generated froma voxel cluster of the undynamic CT which results in the regions 1822,1824, and 1826 are lower than a threshold value and the trajectorysimilarity values for the voxels inside the voxel clusters of theundynamic CT from which the regions 1810 and 1812 are generated in FIG.12 have a value above the threshold value. Again, the aforementionedexample relates to the case where the undynamic CT corresponds to theplanning CT.

Detailed examples for the calculation of trajectory similarity valuesare given below.

FIG. 7 relates to an exemplary embodiment for determining the dynamicDRRs according to the flowchart shown in FIG. 7. According to theflowchart shown in FIG. 7, the computer implemented method relates tothe case where the undynamic CT is the planning CT. For example, thereis a step of selecting one of the sequence CTs as the planning CT andthe undynamic CT. This step can be performed by an operator.

For example, the steps of FIG. 6 are also performed according to anexemplary embodiment described in FIG. 7. The combination of steps ofFIG. 6 are indicated as step S20 in FIG. 7. For example, it can bedefined that the undynamic CT should be the planning CT before or afterstep S20 or simultaneously to step S20 (see step S22 in FIG. 7).

In step S24 the dynamic DRRs are determined by considering thetrajectory similarity values during DRR generation from the planning CT.As mentioned above, the consideration can be performed by modifying theabsorption properties (Hounsfield values) described by the image valuesof the planning CT in dependence on the trajectory similarity valueassigned to the corresponding image element. For instance assume, thetrajectory similarity values related to anatomical elements representedby regions 1822, 1824, 1826, and 1828 are below a threshold, then forexample the image values for these regions are set to black as shown inFIG. 13.

FIG. 8 is a further flowchart which represents at least one furtherexemplary embodiment.

The steps S30 and S32 correspond to steps S20 and S22 in FIG. 7 and canbe interchanged or performed simultaneously.

According to step S34, the dynamic planning CT is determined by usingthe planning CT and the determined trajectory similarity values and bychanging the image values of the planning CT in dependence on thetrajectory similarity values. For example, the image values of theplanning CTs represent Hounsfield values which are a measure for theinteraction of the corresponding anatomical body part represented by theimage value with the x-rays. By changing the image values of theplanning CT in dependence on the trajectory similarity value, thesubsequent determination of the dynamic DRR is influenced. Thisdetermination is performed in step S36. The dynamic DRR is performed inthe usual manner of generating a DRR but not based on a usual planningCT but on the dynamic planning CT determined in step S34.

According to the at least one exemplary embodiment shown in FIG. 9,there is first the step S40 which corresponds to the combination ofsteps shown in FIG. 6. Before, after or simultaneously this step, a stepS42 is performed for acquiring a planning CT independently from the4D-CT. This step is step S42. Based on the undynamic CT determined instep S40, a planning transformation is determined from the undynamic CTto the planning CT for instance by using image fusion. This is done instep S42.

The step S46 can be performed before S42 or step S44 or simultaneouslythereto, for example. The step S46 uses the trajectory similarity valuesdetermined in step S40 to determine the similarity image explainedabove.

According to step S48, the planning transformation determined in stepS44 is applied to the similarity image.

According to step S49, the dynamic DRR is determined by consideringimage values of the transformed similarity image during DRR generationfrom the planning CT. The “consideration of image values” is performedin the same manner as described above with respect to the generationfrom the planning CT in step S24.

According to the at least one exemplary embodiment shown in FIG. 10,which is an exemplary flowchart, a step S50 is performed, whichcomprises the steps of the FIG. 11.

For example, a step S52 is performed, which relates to the acquisitionof the planning CT independently from the 4D-CT. That is, the patient isfor instance before or after the generation of the 4D-CT subjected tomedical image generation by means of an analytical device for generatinga CT. According to at least one exemplary embodiment, the planning CT isstatic and not time dependent.

According to the step S54, a planning transformation is determined fromthe undynamic CT to the planning CT. For example, this is performed inthe manner as described before with respect to step S44.

According to step S56, the similarity image is determined by using thetrajectory similarity values determined in step S50. For example, thestep S56 is performed before or after step S54 or before or after stepS52 or simultaneously to one of those steps.

According to step S57, the planning transformation is applied to thesimilarity image for determining a transformed similarity image.

For example according to a further step S58, the dynamic planning CT isdetermined by using the transformed similarity image. That is, thetrajectory similarity values of image elements of the similarity imageare used to modify image values of corresponding image elements of theplanning CT. “corresponding image elements” are image elements which areat the same position in the planning CT as corresponding image elementsin the similarity image.

For example, in a step S59, the dynamic DRR is determined based on thedynamic planning CT by applying usual methods known in the art fordetermining a DRR from a CT.

According to at least one further exemplary embodiment, a flowchartshown in FIG. 11 describes method steps of the at least one furtherexemplary embodiment. According to step S60, the steps S10 and S12 areperformed. According to step S61 the planning CT is acquiredindependently from a 4D-CT as described above with respect to step S42or step S52. For example, the step S60 is performed before, after orsimultaneously to step S61 or S62.

For example, according to step S62, the planning transformation isdetermined based on the undynamic CT and the planning CT.

For example in a step S63, the steps S14 and S16 of FIG. 6 are performedfor determining the trajectory similarity values. For example, thedetermined trajectory similarity values are used in step S64 todetermine the dynamic CT. The dynamic CT is a three-dimensional imagewhich is for example determined by changing image values of theundynamic CT. The change is performed based on the trajectory similarityvalues determined in step S63. For example, in step S63 the trajectorysimilarity values are determined for particular image elements of theundynamic CT. That is, the trajectory similarity values are assigned tothe respective image elements. The assigned trajectory similarity valuesare then used to change the image values of image elements of theundynamic CT in step S64. For example, this is at least done for atleast a part of the second image elements. For example, this is done incase the trajectory similarity values are below a predeterminedthreshold.

For example, according to another step S65, the dynamic planning CT isdetermined by applying the planning transformation to the dynamic CT.

For example, according to a step S66, the dynamic DRR is determinedbased on the dynamic planning CT in a manner which is usual fordetermining a DRR from a CT.

FIG. 12 has already been described above.

FIG. 13 represents a schematic and exemplary example of a dynamic DRR.It is assumed that the region 1810 represents the treatment body part(for instance tumor). FIG. 13 represents a region which has beengenerated from the planning CT. The region represents the DRR projectionof a voxel cluster. The trajectory similarity values assigned to thevoxel cluster are above a predetermined threshold value. That is, theregion 1812 represents a body part which undergoes a similar vitalmovement as the treatment body part 1810. The term “similar” coversherein identical and the usual meaning of “similar”. For example, imagevalues related to trajectory similarity values above a predeterminedthreshold remain unchanged are not influence by the trajectorysimilarity values, and remain for example unchanged during determinationof the dynamic DRR. In FIG. 13, the regions 1822, 1824, 1826 and 1828are missing since the trajectory similarity values relating to thoseregions are below a predetermined threshold value. According to anexemplary alternative embodiment, the trajectory similarity value is avalue which represents the result of application of the thresholdfunction. That is, the trajectory similarity value is for example abinary value which is for example zero for “non-similarity” and one for“similarity”. That is, in this exemplary embodiment, the trajectorysimilarity values for the voxel clusters which represent the regions1822, 1824, 1826 and 1828 in the planning CT are related to trajectorysimilarity values which indicate non-similarity (for example having avalue of 0).

FIG. 14 shows at least one exemplary embodiment according to an aspectof Annex A which is related to a system. The system comprises forexample a computer 200. To the computer 200 is connected a monitor 201,a keyboard 202, and a mouse 203, for example. For example, the computer200 is connected to the treatment device 100 which can, for example, bemoved along an arc 600. For example, x-ray devices 310 and 320 are usedto make a two-dimensional x-ray image from a patient 400 which is placedon a couch 500. Alternatively or additionally, the computer 200 can beconnected to the couch 500 for changing the position of the couch 500.Alternatively or additionally, the computer 200 can be connected to ananalytical device 330 for generating the 4D-CT. Additionally oralternatively, the computer 200 can be connected to the analyticaldevice 340 for generating the planning CT. The connections describedabove are for example constituted to transfer image data. The connectioncan be wired or wireless.

Exemplary Steps of at Least One Example

According to an example, the different points in time assigned torespective sequence CTs referred to different breathing states of apatient. For example, the respective sequence CTs are assigned to 100%inhaled, 25% exhaled, 50% exhaled, 75% exhaled, 0% inhaled, 25% inhaled,50% inhaled, 75% inhaled.

For example, one of the sequence CTs, to which a particular point intime (for instance particular respiratory state) is assigned, isselected as the undynamic CT. The selection is for instance performed asdescribed in WO 2015/127970. That is, that one of the sequence CTs isselected as undynamic CT, in which the target is good discernible.

For example, in order to determine the primary and secondarytrajectories, image fusion (for example elastic fusion) is performed forthe different points in time (respiratory states).

For example, the undynamic CT acts as a source for the calculation ofthe trajectories. For example, elastic fusion mapping is used to get afirst image element (target point) at a certain point in time (forinstance certain phase of respiration) for every first image element ofthe undynamic image. For example, the image elements are voxels orcluster of voxels.

For example, the trajectory is defined by means of the image elements atdifferent points in time. For example a trajectory is mathematicallydefined by T, then T={source point, target point (10%), target point(20%), . . . , target point (90%)}.

For example, the points of the trajectory describe positions ofthree-dimensional image elements for a particular point in time, forexample of voxels or cluster of voxels. For example, the trajectory is asorted list of the points. For example, the points are sorted by time(for example phase, for example phase of respiration).

Examples for calculating a measure of similarity for the trajectories isgiven in the following.

First example of calculation of a similarity measure is based on a sumof squared differences.

In the following, the abbreviation “SSD” stands for sum of squareddifferences. The abbreviations X, Y, Z stand for the coordinates of athree-dimensional coordination system within which the trajectory isdescribed. The latter T1 stands for example for a trajectory of atreatment body part, for example of an isocenter of the treatment bodypart or of center of mass of a treatment body part. That is T1x(i) isthe x coordinate of the treatment body part at the time (for instancephase) “i”. 1x is the average x coordinate of the treatment body partaveraged over all points in time (for example all states ofrespiration). Correspondingly, T2x stands for the x coordinate of animage element (for example voxel) of the undynamic CT at the point intime (i) and 2x stands for the average x coordinate of this imageelement averaged over the different points in time (for example statesof respiration). The calculation is for example as follows:

${SSDX} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1x}(i)} - {\overset{\_}{T}}_{1x}} \right) - \left( {{T_{2x}(i)} - {\overset{\_}{T}}_{2x}} \right)} \right)^{2}}$${SSDY} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1y}(i)} - {\overset{\_}{T}}_{1y}} \right) - \left( {{T_{2y}(i)} - {\overset{\_}{T}}_{2y}} \right)} \right)^{2}}$${SSDZ} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1z}(i)} - {\overset{\_}{T}}_{1z}} \right) - \left( {{T_{2z}(i)} - {\overset{\_}{T}}_{2z}} \right)} \right)^{2}}$${SSD}_{XYZ} = \frac{{w_{x}*{SSDX}} + {w_{y}*{SSDY}} + {w_{z}*{SSDZ}}}{w_{x} + w_{y} + w_{z}}$

The above equations represent an approach to compute a measure ofsimilarity of trajectories based on sum of squared differences. SSDXYZis an example for a trajectory similarity value or the result ofapplying a threshold function to SSDXYZ is an example for a trajectorysimilarity value.

According to another example, correlation and amplitude correspondenceare determined separately for determining the measure of similarity. Forexample, as described below, the correlation and the amplitudecorrespondence can be mixed, after separate determination in order todetermine a trajectory similarity value as a measure of similarity orcan respectively be used as a measure of similarity.

According to an example, a normalized correlation coefficient iscalculated as follows:

For all three dimensions x, y, z the correlation coefficient is computedseparately and the average correlation coefficient is taken as finalmeasure. One could also think about weighting the correlationcoefficients e.g. if a tumor is moving with diaphragm I-S correlationcoefficient y (I/S) should get more weight. The equations below describecomputing the normalized correlation coefficient for x, y, z, and thecombination to be taken as a trajectory similarity value. T1 and T2 havethe meaning as described above, and n is the number of points of eachtrajectory.

${SSDX} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1x}(i)} - {\overset{\_}{T}}_{1x}} \right) - \left( {{T_{2x}(i)} - {\overset{\_}{T}}_{2x}} \right)} \right)^{2}}$${SSDY} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1y}(i)} - {\overset{\_}{T}}_{1y}} \right) - \left( {{T_{2y}(i)} - {\overset{\_}{T}}_{2y}} \right)} \right)^{2}}$${SSDZ} = {\sum\limits_{i = 1}^{n}\;\left( {\left( {{T_{1z}(i)} - {\overset{\_}{T}}_{1z}} \right) - \left( {{T_{2z}(i)} - {\overset{\_}{T}}_{2z}} \right)} \right)^{2}}$${SSD}_{XYZ} = \frac{{w_{x}*{SSDX}} + {w_{y}*{SSDY}} + {w_{z}*{SSDZ}}}{w_{x} + w_{y} + w_{z}}$

The above equations represent an example for an approached compute asimilarity measure for describing the similarity between trajectoriesbased on correlation coefficient. The abbreviation “CC” stands forcorrelation coefficient. CCXYZ is an example for a trajectory similarityvalue or the result of applying a threshold function to CCXYZ is anexample for a trajectory similarity value.

To determine a trajectory similarity value, a correlation coefficientcan be combined with a value which describes similarity of amplitude oftrajectories. An exemplary approach is described below:

For correlation coefficients that exceed a certain threshold (e.g. 0.7)one could add a second threshold focusing on the amplitude. The moreaccordance in the absolute value of the value, the higher the value.Here an exemplary equation focusing on the main direction of the target,in this case inferior-superior (I-S), the breathing motion caused by thediaphragm.

$A_{IS} = \frac{{Min}\left( {A_{1},A_{2}} \right)}{{Max}\left( {A_{1},A_{2}} \right)}$

In the above equation A1 describes the peak to peak amplitude of atrajectory of the treatment body parts (for example isocenter or centerof mass of treatment body part). For example the amplitude is along aparticular axis of the coordinate system or a long one of the axisdescribed for instance by a rotational ellipsoidal. A2 describes thecorresponding peak to peak amplitude of an image element of theundynamic CT. The terms “Min” and “Max” stand for the function ofdetermining the minimum respectively the maximum of A1 and A2.

According to a further embodiment, the threshold value of the abovedescribed threshold function is changed in dependence on the similarityof amplitudes which is for example described by AIS. AIS is an examplefor an amplitude similarity value.

As described above, the planning CT can be one of the sequence CTs (forexample bins) of the 4D-CT or can be generated separately. In thefollowing, examples for this are described.

A scenario is that the Planning CT is one of the bins of the 4DCT scan.Then, for example, the dynamic image is not registered to the treatmentvolume, that is the planning transformation is not performed. (Remark: A4DCT scan consists of several volumes/bins, each volume/bincorresponding to a specific respiratory state. Typical labeling: 100%Inhaled, 25% Exhaled, 25% Exhaled, 75% Exhaled, 0% Inhaled, 25% Inhaled,25% Inhaled, 75% Inhaled).

In case the Planning CT is not part of the 4DCT scan, the planning CT isregistered to one of the sequence CTs (by using the planningtransformation). The registration procedure and thus the determinationof the planning transformation would mean for example a first rigidregistration step (concentrating e.g. on bones) yielding atransformation that brings the two in a common coordinates system,followed by a second deformable registration yielding a secondtransformation which represents a deformation field. The combination ofthe first and second transformation represents an example for a planningtransformation. The question which one of the sequence CTs to be used asundynamic CT:

-   -   If the planning CT was taken during a specific breathing phase        one could register the planning CT to the sequence CT which        corresponds to the same respiratory state.    -   One could also register consecutively to all sequence CTs, and        select the most similar sequence CT as the undynamic CT. ‘Most        similar’ could for instance mean selecting the registration that        resulted in the fewest deformation around the target area.    -   Or as mentioned above, one could select that one of the sequence        CTs in which the treatment body part is best discernable.    -   Or a combination of the above.

According to an example, the computer implemented method is constitutedto display the dynamic DRRs in dependence on selected thresholds. Inparticular, the computer implemented method can be constituted that auser changes the threshold while getting immediate feedback of theeffect of change of threshold by displaying the dynamic DRR. In moredetail, this is for example as follows:

The computer implemented method can be constituted to display a page fordefining the dynamic DRR. This page provides e.g. a slider enabling theuser to set a certain threshold value used by the above describedthreshold function. A first page can show a very strict thresholdresulting in a dynamic DRR nearly containing the treatment body part(target) only. Only voxels following exactly the same trajectory(normalized) are taken into account for rendering. In another page, thethreshold can be decreased and thus more voxels—voxels whose trajectoryis “very similar” to the target—are used for rendering the dynamic DRR.With respect to the Figures showing flowcharts, generally, the sequenceof the steps is not obligatory but just an example. The only requirementis that data necessary for a determination step have to be acquiredbefore the respective determination.

Different Aspects According to Annex A

According to a first aspect, a computer implemented method fordetermining a two dimensional DRR is disclosed referred to as dynamicDRR based on a 4D-CT, the 4D-CT describing a sequence of threedimensional medical computer tomographic images of an anatomical bodypart of a patient, the images being referred to as sequence CTs, the4D-CT representing the anatomical body part at different points in time,the anatomical body part comprising at least one primary anatomicalelement and secondary anatomical elements, the computer implementedmethod comprising the following steps:

-   -   acquiring (S10) the 4D-CT;    -   acquiring (S22, S32, S52, S61) a planning CT, the planning CT        being a three dimensional image used for planning of a treatment        of the patient, the planning CT being acquired based on at least        one of the sequence CTs or independently from the 4D-CT,    -   acquiring (S12) a three dimensional image, referred to as        undynamic CT, from the 4D-CT, the undynamic CT comprising at        least one first image element representing the at least one        primary anatomical element and second image elements        representing the secondary anatomical elements;    -   acquiring (S14) at least one trajectory, referred to as primary        trajectory, based on the 4D-CT, the at least one primary        trajectory describing a path of the at least one first image        element as a function of time;    -   acquiring (S14) trajectories of the second image elements,        referred to as secondary trajectories, based on the 4D-CT;    -   for the image elements of the undynamic CT, determining (S16)        trajectory similarity values based on the at least one primary        trajectory and the secondary trajectories, the trajectory        similarity values respectively describing a measure of        similarity between a respective one of the secondary        trajectories and the at least one primary trajectory;    -   determining (S24, S36, S49, S59, S66) the dynamic DRR by using        the determined trajectory similarity values, and, in case the        planning CT is acquired independently from the 4D-CT, further        using a transformation referred to as planning transformation        from the undynamic CT to the planning CT, at least a part of        image values of image elements of the dynamic DRR being        determined by using the trajectory similarity values.

According to a second aspect, the computer implemented method accordingto aspect 1 is disclosed, wherein image values of image elements of thedynamic DRR are determined in dependence on the trajectory similarityvalues used for determining the image elements.

According to a third aspect, the computer implemented method accordingto one of the preceding aspects is disclosed,

-   -   wherein the undynamic CT is the planning CT (S22, S32); and    -   wherein the the step of determining the dynamic DRR comprises at        least one of the following steps a) or b):        a) determining (S24) the dynamic DRR by using the planning CT        and the determined trajectory similarity values, wherein, during        determination of the dynamic DRR from the planning CT, the        trajectory similarity values are considered; or        b) determining (S34) another three dimensional image, referred        to as dynamic planning CT by using the planning CT and by        changing image values of the planning CT in dependence on the        trajectory similarity values, and determining (S36) the dynamic        DRR by digitally reconstructing the two-dimensional image from        the dynamic planning CT.

According to a fourth aspect, the computer implemented method accordingto one of the aspects 1 to 3 is disclosed, wherein the step of acquiring(S42, S52) the planning CT independently from the 4D-CT is performed andfurther comprising the steps of:

-   -   determining (S44, S54) the planning transformation;    -   acquiring (S46, S56) a three dimensional image referred to as        similarity image from the determined trajectory similarity        values related to the image elements of the undynamic CT;    -   applying (S48, S57) the planning transformation to the        similarity image;        wherein the step of determining the dynamic DRR comprises at        least one of the following steps a) or b):        a) determining (S49) the dynamic DRR by using the planning CT        and the determined trajectory similarity values, wherein, during        determination of the dynamic DRR from the planning CT, image        values of the transformed similarity image are considered; or        b) determining (S58) another three dimensional image, referred        to as dynamic planning CT by changing image values of the        planning CT in dependence on the corresponding trajectory        similarity values of the transformed similarity image and        determining (S59) the dynamic DRR by digitally reconstructing        the two-dimensional image from the dynamic planning CT.

According to a fifth aspect, the computer implemented method accordingto one of aspects 1 to 3 is disclosed, wherein the step of acquiring(S61) the planning CT independently from the 4D-CT is performed andfurther comprising the steps of:

-   -   determining (S62) the planning transformation; and        wherein the step of determining the dynamic DRR comprises:    -   determining (S64) a three dimensional image, referred to as        dynamic CT by changing image values of at least a part of at        least the second image elements of the undynamic CT in        dependence on the trajectory similarity values determined for        the respective image elements;    -   determining (S65) a three dimensional image referred to as        dynamic planning CT by applying the planning transformation to        the dynamic CT; and    -   determining (S66) the dynamic DRR by digitally reconstructing        the two-dimensional image from the dynamic planning CT.

According to a sixth aspect, the computer implemented method accordingto one of the preceding aspects is disclosed wherein

the step of acquiring the primary and secondary trajectories comprises:

-   -   acquiring at least the at least one first image element from the        undynamic CT;    -   acquiring the second image elements from the undynamic CT;    -   determining transformations referred to as sequence        transformations which are constituted to transform the undynamic        CT to one or more of the sequence CTs and/or to transform one of        the sequence CTs to another one of the sequence CTs;    -   determining the trajectories of the at least one first image        element and of at least some of the second image elements by        applying the determined sequence transformation to the at least        one first image element and the at least some of the second        image elements.

According to a seventh aspect, the computer implemented method accordingto one of the preceding aspects is disclosed, comprising a step ofcalculating trajectory similarity values as a measure of similaritybetween trajectories, the step comprising one of the following:

a) determining the respective trajectory similarity values as a functionof positional differences between a first position of the at least onefirst image element defined by the at least one primary trajectory fordifferent points in times and an average of the first position for thedifferent points in time and a positional difference between a secondposition of a respective one of the second image elements defined by thesecondary trajectory for the different times and an average of thesecond position for the different points in time,b) determining correlation coefficients describing a correlation betweenthe trajectoriesc) determining a normalized correlation describing a normalizedcorrelation between the trajectoriesd) determining amplitudes of the trajectoriese) a combination of one of steps a) to c) with d)

According to a eighth aspect, the computer implemented method of one ofthe preceding aspects is disclosed, wherein an anatomic atlas is usedaccording to at least one of the following steps:

at least one of the second image elements are determined by means ofsegmentation using the anatomic atlas; or

for one or more of the second image elements no trajectories aredetermined in dependence on the result of the segmentation achieved bymeans of the anatomic atlas; or

trajectory similarity values related to one or more of the second imageelements are determined in dependence on the result of thedetermination.

According to a ninth aspect, the computer implemented method accordingto one of the preceding aspects is disclosed comprising a display of asuperposition of the dynamic DDR over a two-dimensional X-ray imageand/or aside the two-dimensional X-ray image.

According to a tenth aspect, a computer implemented method fordetermining a three dimensional image referred to as similarity based ona 4D-CT is disclosed and/or for determining a two-dimensional DRRreferred to as dynamic DRR and/or for determining a three-dimensionalimage referred to as dynamic CT, the 4D-CT describing a sequence ofthree dimensional medical computer tomographic images of an anatomicalbody part of a patient which represent the anatomical body part atdifferent points in time, the images being referred to as sequence CTs,the anatomical body part comprising at least one primary anatomicalelement and secondary anatomical elements, the computer implementedmethod comprising the following steps:

-   -   acquiring the 4D-CT;    -   acquiring a three dimensional image, referred to as undynamic        CT, from the 4D-CT, the undynamic CT comprising at least one        first image element representing the at least one primary        anatomical element and second image elements representing the        secondary anatomical elements;    -   acquiring at least one trajectory, referred to as primary        trajectory, based on 4D-CT, the at least one primary trajectory        describing a path of the at least one first image element as a        function of time;    -   acquiring trajectories of the second image elements, referred to        as secondary trajectories, based on the 4D-CT;    -   for the image elements of the undynamic CT, determining        trajectory similarity values based on the primary trajectory and        the secondary trajectories, the trajectory similarity values        respectively describing a measure of similarity between a        respective one of the secondary trajectories and the at least        one primary trajectory; and further comprising at least one of        the following steps:        a) determining the similarity image by determining the        trajectory similarity values to be image values of image        elements of the similarity image, referred to as similarity        image elements; and        optionally displaying the similarity image; or        b) determining the dynamic DRR by using the determined        trajectory similarity values, at least a part of image values of        image elements of the dynamic DRR being determined by using the        trajectory similarity values; and optionally displaying the        dynamic DRR; or        c) determining the dynamic CT by changing image values of at        least a part of at least the second image elements of the        undynamic CT in dependence on the trajectory similarity values        determined for respective image elements and optionally        displaying the dynamic CT.

According to a eleventh aspect, the computer implemented method of thetenth aspect is disclosed, comprising the step of acquiring a planningCT, the planning CT being a three dimensional image used for planning ofa treatment of the patient, the planning CT being acquired based on atleast one of the sequence CTs or independently from the 4D-CT; and

the positions of the similarity image elements correspond to thepositions of the image elements of the undynamic CT to which thetrajectory similarity values are respectively related;

and optionally, in case the planning CT is acquired independently fromthe 4D-CT, further determining a transformation from the undynamic CT tothe planning CT and applying the transformation to the similarity imagebefore displaying the similarity image.

According to a twelfth aspect, a program is disclosed which, whenrunning on a computer or when loaded into a computer, causes thecomputer to perform the method according to any one of the precedingaspects and/or to and/or a signal wave, in particular a digital signalwave, carrying information which represents the program, in particular,the aforementioned program in particular comprises code means adapted toperform all the steps of the method of one of the preceding aspects.

According to a thirteenth aspect, a computer-readable program storagemedium on which the program according to the twelfth aspect is stored,for example in a non-transitory manner.

According to a fourteenth aspect, a computer is disclosed, the computercomprising the compute-readable program storage medium of the thirteenthaspect.

According to a fifteenth aspect, a system is disclosed, comprising: thecomputer (200) of the preceding aspect; and at least one of thefollowing:

a) a display device (201) for displaying the dynamic DRR and aninterface for receiving the 4D-CT; or

b) a couch (500) for placing a patient (400) and a treatment device(100) constituted to emit a treatment beam; or

c) an analytical device (310, 320) constituted for generatingtwo-dimensional x-ray images;

d) an analytical device (330) constituted for generating the 4D-CT; or

e) an analytical device (340) constituted for generating the planningCT.

The invention claimed is:
 1. A computer implemented method comprising:using a dynamic anatomic atlas for matching a patient image and an atlasimage or for matching two patient images; using information on a dynamicproperty of at least one atlas segment as a constraint for the matching;wherein the dynamic anatomic atlas includes static atlas data describingatlas segments; and dynamic atlas data comprising the information on adynamic property which information is respectively linked to the atlassegments.
 2. The method of claim 1 further comprising: acquiring thestatic atlas data and the dynamic atlas data of the dynamic anatomicatlas, the static atlas data describing a static atlas image of theatlas segments; wherein the dynamic anatomic atlas comprises an atlassegment subdivided into atlas subsegments respectively linked withdifferent dynamic properties while exhibiting the same segmentrepresentation information acquiring static patient data describing astatic patient image of a patient segment; matching the static patientimage with the static atlas image; determining the corresponding atlassegment corresponding to the patient segment based on the matching;determining subsegments within the patient segment based on the atlassubsegments of the corresponding atlas segment.
 3. The method of claim 1wherein the information on the dynamic property describes correlationsbetween the dynamic properties of different ones of the atlas segments.4. The method of claim 3, further comprising: calculating based on theinformation on the dynamic properties linked to different patientsegments correlations between the dynamic properties of the differentpatient segments for determining the correlations; and calculating basedon at least the information on the dynamic property linked to a patientsegment, at least one normalized dynamic property for the patientsegment for determining the at least one normalized dynamic property. 5.The method of claim 1 wherein the information on the dynamic propertydescribes at least one normalized dynamic property of at least one atlassegment.
 6. The method of claim 1 further including classifying at leastone dynamic property linked to an atlas segment according to patienttypes.
 7. The method of claim 6, further comprising: enabling ananalysis of an anatomic dynamic of a patient by: acquiring the staticatlas data and the dynamic atlas data, the static atlas data describinga static atlas image; acquiring static patient data describing a staticpatient image of a patient segment; acquiring dynamic patient datacomprising information on a dynamic property, the information beingrespectively linked to the patient segment; matching the static patientimage with the static atlas image; determining a corresponding atlassegment corresponding to the patient segment based on the matching;comparing both the information on the dynamic property linked to thecorresponding atlas segment and the information on the dynamic propertylinked to the patient segment.
 8. The method of claim 1 wherein thedynamic anatomic atlas comprises information on a distribution of atleast one dynamic property.
 9. The method of claim 8, furthercomprising: acquiring the static atlas data and the dynamic atlas data;comparing at least one classified dynamic property of the correspondingatlas segment with the dynamic property of a patient segment; anddetermining based on the comparison, the type of the patient.
 10. Themethod of claim 1 wherein the dynamic anatomic atlas comprises an atlassegment subdivided into atlas subsegments respectively linked withdifferent dynamic properties while exhibiting the same segmentrepresentation information.
 11. The method of claim 1 wherein thedynamic anatomic atlas is generated by: acquiring, based on the staticatlas data a static atlas image of the atlas segments; acquiring staticpatient data describing a static patient image of a patient segment;acquiring dynamic patient data comprising information on a dynamicproperty, the information being respectively linked to the patientsegment; matching the static patient image with the static atlas image;determining a corresponding atlas segment corresponding to the patientsegment based on the matching; generating the dynamic anatomic atlas,the information on the dynamic property linked to the correspondingatlas segment is determined based on the information on the dynamicproperty linked to the patient segment.
 12. The method of claim 11,wherein, based on subregions exhibiting different dynamic propertieswithin the corresponding patient segment, atlas subsegmentscorresponding to the dynamic subregions are determined.
 13. The methodof claim 1, further comprising: enabling an analysis of an anatomicdynamic of a patient by: acquiring the static atlas data and the dynamicatlas data, the static atlas data describing a static atlas image;acquiring static patient data describing a static patient image of apatient segment; acquiring dynamic patient data comprising informationon a dynamic property, the information being respectively linked to thepatient segment; matching the static patient image with the static atlasimage; determining a corresponding atlas segment corresponding to thepatient segment based on the matching; comparing both the information onthe dynamic property linked to the corresponding atlas segment and theinformation on the dynamic property linked to the patient segment. 14.The method of claim 1, wherein the dynamic property is at least one ofthe following: dynamic spatial information, comprising at least one ofinformation on a change of position of an object or information on achange of geometry of an object; dynamic thermodynamic information,comprising at least one of information on a change of temperature of anobject or information on a change of pressure of an object orinformation on a change of volume of an object; fluid-dynamicinformation, comprising at least one of information on a change of fluxor information on a change of velocity of a substance within an objector information of a change of density of a substance within an object,wherein the object is at least one of a patient segment or a subsegmentthereof or one of the atlas segments or a subsegment thereof.
 15. Atleast one non-transitory computer readable storage medium comprising:instructions stored on the storage medium that, in response to executionof the instructions by one or more processors, cause the one or moreprocessors to: match a patient image and an atlas image or for matchingtwo patient images; constrain the match of the two images usinginformation on the dynamic property of at least one atlas segment;wherein a dynamic anatomic atlas includes static atlas data describingatlas segments and dynamic atlas data comprising the information on adynamic property which information is respectively linked to the atlassegments.
 16. At least one computer, comprising: at least one processorand a memory, wherein the at least one computer includes at least onenon-transitory computer readable storage medium comprising instructionsthat, in response to execution of the instructions by the at least oneprocessor, cause the at least one processor to: match a patient imageand an atlas image or for matching two patient images; constrain thematch of the two images using information on the dynamic property of atleast one atlas segment; wherein a dynamic anatomic atlas includesstatic atlas data describing atlas segments and dynamic atlas datacomprising the information on a dynamic property which information isrespectively linked to the atlas segments.